Sunday, 27 November 2011

Cryptoscan: Fixed Windows Vista+ support

There’s been a minor update to the batch script that I’ve provided with Cryptoscan as previous version was limited to only work with versions of Windows supported by Volatility 1.3 beta. I know that to truly solve the problem I should have ported the module to Volatility 2.0 but I already tried it and I miserably failed. Anyone wants to help? ;>

The new batch script runs Cryptoscan using version 1.3 and then the other two modules (i.e. Strings and Modules) using version 2.0.

To get it working here are 3 simple steps:
1.       Extract 'Volatility-1.3_beta'.
2.       Extract 'Volatility-2.0.standalone' to the same folder as before.
3.       Extract 'Cryptoscan' to the same folder too (overwrite if asked).

You can run ‘Cryptoscan.cmd’ now and enjoy! ;)

If you have some problems getting it working then check my previous post or leave a comment.

PS. There's been a small change to the provided binaries. Instead of using GnuWin32, the batch script uses UnxUtils since they do exactly the same job and are smaller in size.

Tuesday, 22 November 2011

Compressed (and Encrypted) Alternate Data Streams

The technique of hiding data within NTFS Alternate Data Streams (aka ADS) has been widely documented and it isn’t something that could take a forensic investigator by surprise. Then about a month ago I came across an article with an extra-ordinary name – Stealth Alternate Data Streams and Other ADS Weirdness, which hinders certain tools from finding the ‘hidden’ streams. You can probably see where I’m going with this… today I read an article titled Detecting and Manipulating Compressed Alternate Data Streams in a Forensics Investigation, unfortunately there's no freely available copy of it, which describes another technique of hiding data.

The article explains that by compressing a file using WinRAR with an option for preserving NTFS streams turned on (Figure 1), which according to this post is the only archiver that supports this technique at the moment, we can evade some of the anti-virus scanners. I also tested it on 7-Zip and WinZip 16.0 but neither of them can handle file streams.

Figure 1. Preserving ADS.

The article, not surprisingly, uses VirusTotal to compare results of scanning the archive by different anti-virus engines. In October 2007, when the research was carried out, only four out of all the supported scanners detected the embedded malicious software. Bearing in mind that it was a few years ago, I couldn’t resist trying it out myself. I firstly created a text file with some random text and then appended a file with an EICAR-TEST file to it. Then I archived the file making sure it includes file streams, and scanned it with an anti-virus which was on my machine but it didn’t detect anything (I used F‑Secure Internet Security 2011 with today’s virus definitions).

Figure 2. The result of scanning a RAR archive with an EICAR-TEST file hidden in a file stream.

I was also curious about how would other anti-virus scanners cope so I uploaded the file to VirusTotal and the detection rate was only 37.2% (Figure 2). But it gets even better… you see, the other WinRAR’s NTFS option reads ‘save file security’. The article doesn’t cover this feature, but I decided to give it a go: I encrypted the file (with a stream) using the NTFS Encryption (aka EFS) feature and then archived it making sure that both of the WinRAR's NTFS options were selected. Guess what, the detection rate was 0% (Figure 3).

I opened the archive with WinRAR on different machine (VM with Windows XP SP3) and the file was in green indicating that it is compressed but when I unpacked it, because the user didn’t have the right encryption key, WinRAR skipped the streams. What was even more surprising, the file extracted without showing any errors what‑so‑ever. Just to clarify, the archive successfully extracts files with streams when opened by the user who created it. It indicates to me that for some weird reason WinRAR has different implementation of compressing encrypted files and their streams: it decrypts files and keeps them in "plain-text" unlike file streams which are stored encrypted.

Figure 3. The result of scanning a RAR archive with an encrypted file stream.

Let’s consider the implications of it. If a suspect had used a RAR archive to hide NTFS Encrypted files then unless the investigator tried to open the archive under the suspect’s account (e.g. using LiveView), or imported the suspect's encryption key on his investigation workstation (I’m not quite sure how and if it’s doable), then he’d have missed the streams. On the other hand, if the archive is suspiciously large compared to its content, it may be a clue to an investigator.

Update (21 Dec 2011): I decided to write a tool that would scan a hard drive for any RAR files to see if they contain any NTFS streams. Before I did it though, I tried again the above-mentioned  method with encrypted NTFS streams. As it turned out, I must have made a mistake when firstly testing this method because WinRAR has a bug which makes it skip NTFS streams of encrypted files. Thus, the 0% detection rate from VirusTotal (because the file wasn't included!).

Update (31 Jan 2012): I've contacted the team behind WinRAR about the issue and this is their response: "Sorry, WinRAR does not support encrypted streams now. It would require another approach like using ReadEncryptedFileRaw instead of BackupRead for encrypted files and it is not implemented. Maybe we'll do it in the future." So as one can infer, it is a known issue and may, or may not, be solved in the future.

Friday, 11 November 2011

File System Tunnelling

While reading through Harlan Carvey’s slides from PFIC 2011 about Introduction to Windows Forensics, I came across a term File System Tunnelling. A quick Google search directed me to Raymond Chen’s blog post titled The apocryphal history of file system tunnelling.

As I started reading, I immediately realised that I have heard about this concept before but up until then I didn’t know what it was. Without spoiling it for you, I won’t tell you exactly what it says, but I really liked the analogy of quantum mechanics and how the process of “short name saving” works.

Cutting long story short, even if you know what File System Tunnelling is but you don’t know what I’m talking about, go and give it a read!

Monday, 31 October 2011

Memory Forensics: Presentation

This Wednesday I’m going to give a 20 minutes presentation about Live Forensics with a focus on Memory Forensics.

I know it may sound a bit strange but I’ve heard about Death by PowerPoint so many times that I decided to run it differently: I asked students to read a chapter 3 from Harlan Carvey’s book WFA 2e and to read the slides before turning up for the lecture, and then I’ll do a ‘Questions and Answers’ session followed by a demo.

The slides can be found here and this is the list of ideas for the conversation:
  • When? 2005, DFRWS (digital forensics research workshop)
  • Why? Passwords/encryption keys, hidden stuff by rootkits, encrypted/obfuscated malware
  • What’s the order of live forensics? Why is imaging memory the first? What was the Locard’s Exchange Principle about, again?
  • Halt a process/system when imaging?
  • Other copies of memory? Hibernation, crash dump, swap space.
  • What is virtual memory?
  • What is the default size of a page in memory? 4KB (4096 bytes)
  • What’s wrong with \Device\PhysicalMemory? When did it happen?
  • Are there any other methods?
  • The limit of collecting up to 4GB of RAM?
  • Some tools may skip the page file. What it means? Why? The switch ‘-page’.
  • F-Response, completely different approach. Why?
  • Hardware ways of dumping memory? What does PCI stand for? Payment Card Industry? NOT!
  • What about cloud computing?
  • How to generate a crash dump? What happens behind the scenes? What’s the size limit? What are the two requirements? What are the concerns?
  • Is RAM wiped at boot?
  • Digital Corpora Project.
  • Offensive Computing, over 3mln malware samples.

By the way, can anyone tell me what does ‘PFN Mapping’ in Win32dd do?

Update (05 Nov 2011): Firstly, a demo of using Cryptoscan v2.0 is now available on YouTube. Secondly, I've received an answer to the question above (thanks to Andrew Case): supposedly, this technique doesn't rely on Windows API and hence, the memory footprint is smaller. Andrew also explained that PFN Mapping "enumerates all the physical pages of RAM (non-hardware addresses) and then reads and writes them out directly."

Sunday, 30 October 2011

Storing Binary Files in the Registry

As part of a group work at university, my colleagues and I are in the process of preparing a computer image for the first year students to investigate it as a part of their assignment. The main task for us is to pretend to be a “bad guy” and hide numerous files on that image using as many techniques as possible.

The technical level of these methods should vary and if some of them happen to be too challenging then to leave some clues behind (e.g. browser’s history or source code), so that with basic investigation knowledge and help of omniscient Google they shouldn’t have a problem to find them.

So far we had about 20 different ideas –ranging from hiding data in NTFS bad sectors, file slack, and Ms Word documents to pseudo-encryption algorithms and file merging. It’s been very interesting so far and I’m glad with the way our team works: we all have learnt a lot just by researching different ideas and developing various tools, and then by sharing the findings with each other. Hence, by the way, my last post about recovering the passphrase to a TrueCrypt volume from a memory dump.

One of the remaining things on my list for this group work is to investigate whether it is possible to store binary files, such as images, in the Windows registry.

I did some Googling, as always, but exceptionally I couldn’t find much, besides a couple of articles about storing data in a REG_BINARY value type. Thus, I decided to develop one myself and after a few hours I had a ready “framework” for reading a file to memory, transforming it and then saving to the registry, and the other way around of course.

The tool is called bin2reg and at the moment supports two basic encoding mechanisms: XOR against a password, and Windows Data Protection API (DPAPI). If anyone wanted to treat it seriously though, then the latter is obviously a better solution.

There’s a small limitation at the moment: the application can only store files smaller than 1MB as this is the maximum size for any single Registry value. It can be easily solved by splitting a file into smaller parts and then saving them separately to the registry.

Here’s the code written in C#, so you need to have .NET Framework to run it, and a compiled binary for those who don’t have Visual Studio.

Monday, 24 October 2011

Updated Cryptoscan for the Volatility Framework

This post is about an updated version of Cryptoscan which makes it compatible with the latest versions of TrueCrypt.

Cryptoscan is a module for the Volatility framework which scans a memory image for TrueCrypt passphrases. For a passphrase to be stored in a memory by TrueCrypt, an option “Cache passwords and keyfiles in memory” needs to be selected. Consequently, the credentials are saved until a user wipes the cache or restarts the computer.

I tried Cryptoscan version 1.0, released by Jesse Kornblum in 2008, and despite my numerous attempts I failed to recover a passphrase. Then I decided to study its source code and this is what caught my eye:

    Passphrases are stored in a structure containing a magic
    value (0x7d0), a passphrase length, and then 64 bytes of
    passphrase data. The data must contain exactly length ASCII
    characters and all remaining bytes must be zeros.

After that I searched a previously created memory dump for my passphrase and the magic value. Strangely enough, I had no problem finding the passphrase but there was no value preceding it.

As I was puzzled with the lack of magic value, I thought that maybe TrueCrypt’s source code could reveal anything useful. Spot on! The Password header file from the version 7.1, the latest at the time of writing this post, shows the following data structure:

    typedef struct
          // Modifying this structure can introduce incompatibility with previous versions
          unsigned __int32 Length;
          unsigned char Text[MAX_PASSWORD + 1];
          char Pad[3]; // keep 64-bit alignment
    } Password;

This proves to me that maybe there used to be a magic value before the password’s length in some of the previous versions, but definitely there isn’t one anymore. I have looked through different versions of the code and it seems that the structure has been introduced with the version 4.0 and then 5.0 added the padding.

So the comment from Cryptoscan could now look something like this:

    Passphrases are stored in a structure containing a passphrase
    length (a value between 1 and 64 stored in the first of the four
    bytes),65 bytes of passphrase data and then 3 bytes of padding
    to keep 64-bit alignment. The data must contain exactly length
    ASCII characters, all remaining bytes must be zeros except for
    the padding which has a random value.

If we know that there’s no magic value anymore, then modifying the code to search for passphrases according to the new requirements would cause a flood of strings which aren’t necessarily related to TrueCrypt’s “black magic”. This is an example of running modified Cryptoscan on Windows XP SP3.


To reduce the number of false positives, we can extract strings along with their offsets and then link them to a process they belong to, which can be done using Volatility’s Strings module. Once we have that, we can ignore all the strings that belong to other processes than TrueCrypt (on Windows it’s a module, TrueCrypt.sys). The final result is surprisingly good and it can be easily automated.

To prove that it actually works I created a Windows batch script that uses the updated version of Cryptoscan and Win32 ports of awk, grep and sed to filter the output. To get it working, extract the archive to a folder with Volatility 1.3 and run the batch script.

An archive with the script and the necessary binaries can be downloaded from here.

Update (27 Nov 2011): I've received an email from one of the students at university (thanks Phil!) complaining that the script won't work with Windows Vista (and higher for that matter). I've had a look and he's got a point, the script only works with Volatility 1.3 beta which is limited to older versions of Windows. I've updated the batch script to accommodate to that. You can read more on that in this post.

Thursday, 20 October 2011

Running Cryptoscan on Windows XP

This post will cover how to get Volatility 1.3 beta running with Cryptoscan plug-in on Windows XP.

I've been currently working on recovering a passphrase to an encrypted volume from a memory dump. The memory dump was taken using MoonSols DumpIt on Windows XP SP3 with TrueCrypt v7.1. To do so I used Cryptoscan written by Jesse Kornblum (aka Computer Forensics Research Guru). Thanks to Jesse for the great work!

I’ve encountered some problems while trying to get it running because, as it turned out later, Windows’s built-in ZIP extractor missed some of the files when I closed the explorer’s window. So if you’re getting any errors, I recommend trying to extract it all over again and making sure you have Python 2.6 as opposed to version 2.7 (from my experience it works on both though).

Once you copy the to the folder memory_plugins, you’re ready to go.

To test it out run a command prompt, enter the folder with your copy of Volatility and type in the following command:

    C:\Volatility-1.3_Beta> python volatility cryptoscan --help
    Usage: cryptoscan [options] (see --help)

      -h, --help            show this help message and exit
      -f FILENAME, --file=FILENAME
                            (required) XP SP2 Image file
      -b BASE, --base=BASE  (optional, otherwise best guess is made) Physical
                            offset (in hex) of directory table base
      -t TYPE, --type=TYPE  (optional, default="auto") Identify the image type
                            (pae, nopae, auto)

If instead of the message above you’re getting an error saying "'python' is not recognized as an internal or external command, operable program or batch file." then it probably means you haven’t set up the environmental variable Path correctly (page 4 of installation document). You can check it by writing 'echo %Path%' in a command prompt.

That’s it for now. If you have any questions then leave a comment.

In the next post I’ll cover how to update Cryptoscan to get it working with the current version of TrueCrypt.


This is my first post so don’t be too harsh on me.

I’ll start from saying it’s been absolutely ages that I wanted to create a blog, a place where I could store all my reflections on my passion to computers and especially Information Security (IS), and I’m glad I’ve finally managed to do so. Weirdly enough, the biggest obstacle for me wasn’t coming up with a post (quite the opposite) but thinking of a catchy title that would accurately reflect the nature of the blog.

Having said that, I’d like to share my thoughts on everything I find worth enough to write about, regardless of whether it’d be a programming, computer forensics, penetration testing or a goat browsing the Internet.

Finally, I need to mention that I really look up to the F-Secure’s blog for the style of writing and I hope to that mine won’t be too far from that.

Enjoy the reading!