Joebox

Comparing existing sandbox applications

Currently three malware behaviour analysis applications exist. CWSandbox (University of Mannheim), Anubis (Technical University of Vienna) and Joebox (private one man project).

• Joebox
• CWSandbox
• Anubis

Each software has his own unique concept. Because no detailed comparison exists I decided to evalute all applications which currently analyse malware for free under the following technical aspects:

• Function intercepting technology
• Analysis completeness
• Used environment
• Pointer handling

Function intercepting technology
Normally a sandbox application has to intercept system calls and log them in a human or machine readable format (HTML or XML). On windows based operating system there are many techniques to attain this goal. For each existing sandbox the function intercepting technology is explained in detail.

• Joebox
As described in joebox's concept the application uses detour API hooking. After some specified API functions 5 or more bytes are overwritten with a jump to an analysis function. This function itself contains a part to analyse the function arguments and a jump to a tramplone function which calls the original API function. The trampoline function is used furthermore to store the overwritten bytes.
Strengths
• Easy to implement
• API function arguments and the function return value can be analysed
Weaknesses
• Easy to detect therefore actions needed to prevent detection
• Driver activities can not be intercepted and analysed


• CWSandbox
The application uses API detour hooking too. For further details see technical details.


• Anubis
Anubis does not use well-known windows hooking techniques. The application excutes the binaries on an emulated software environement called QEMU (no virtualisation QEMU is completly software based). QEMU emulates a lot of different computer systems and is published under GPL/BSD license. The anubis team changed the implementation of QEMU to easly controll the execution and detect API function calls by analysing the emulated CPUs register and the virtual memory.
Strengths
• Large control over code execution
• No limited access on virtual memory
• Easy analyse API function arguments and return value
Weaknesses
• Very complex technique
• Possible to detect QEMU presence (some symantec papers have been released about the detection of software emulation by Peter Ferrie)
• Difficult to prevent detection
• Driver activities can not be analysed

Analysis completeness
To compare the completeness of analysis reports I tested a very simple keylogger on each sandbox. The keylogger listens for keyboard input. If he receives some keys he stores them on the local harddisk. All sandboxes should report that a keylogger has been installed. In addition I have sent a basic DLL injection binary to each sandbox. Also I tested a binary which uses a native API function.

• Joebox
Keylogger analysis
Dll injection analysis
Native Dll load analysis
Average of completeness
• Keylogger: nearly 100%, we see that a keyboardhook has been installed by SetWindowsHookEx and information has been stored on the local harddisk


• Dll injection: 80% there is an overview about used injection technique and loaded dlls provided, but no information about which target process


• Native Dll load: 0% joebox currently does not support logging native API function, see joebox api definition.


• CWSandbox
Keylogger analysis
Dll injection analysis
Native Dll load analysis

Average of completeness
• Keylogger: 0% there is no relevant information provided, you can not see any keyboard installation


• Dll injection: 100% code injection detected (explorer.exe), information about loaded Dlls and used techniques provided


• Native Dll load: 80% DLL seeable in dll handling section, but no information about load function provided


• Anubis
Keylogger analysis
Dll injection analysis
Native Dll load analysis

Average of completeness
• Keylogger: 0% there is not at all any relevant information provided


• Dll injection: 80% both injected Dlls have been detected, but no relevant information about the injection technique is provided


• Native Dll load: 80% DLL loading seeable (run-time Dll section), but no information about load mechanisme provided

Used environment
Normally a sandbox application analyses binaries on a virtual or emulated machine or one real computer system. Virtual and emulated environement can be easily detected by malware but the code execution controll and the restore possiblies are better than a real system.
Always a packet sniffer should be used on extra machine to capture the traffic generated by the exectued binary. This is important because there is no possiblity to capture all packets on the sandbox machine self (there will be always a way to phone home).

• Joebox
Joebox uses a real machine, so there will be limited ways to detect the sandbox. Furthermore joebox contains a sniffer component located on a different machine, which is able to sniff independent from the executing machine the complete network traffic.
Strengths
• Difficult to detect presence of analysis environments
• Possibilty to log 100% of all generated network traffic
• No additional virtualisation and emulation software needed
• Installed operating system is Windows Vista
Weaknesses
• Needs a complex and slow restore technique


• CWSandbox
The CWsandbox system uses the virtualization software VMware to exectue binaries. There are many well documented ways to detect the vmware environment. Often malware uses such detection to avoid analysis which is really a fact.
Strengths
• Fast and simple restore, works with snapshots
Weaknesses
• Software licence needed
• VMware can be detect very simply
• The sandbox is not able to analyse the whole network traffic (use of a socket driver or bypassing detour hooking)


• Anubis
As previously discused Anubis usse a system emulation environment called QEMU. QEMU is able to emulate not only the CPU but also the complete periphery.
Strengths
• Fast restore
• Simple restore, work with snapshots
• QEMU is published under the GPL/BSD license
• Possiblity to capture the complete network traffic through QEMU virtual network card
Weaknesses
• QEMU can be detected and exploited see papers from Tavis Ormandy
• Complex environement

Pointer handling
During the implementation of joebox, I got into deep troubles. The main problem was the pointer argument analysis of called API function. Because there are thousands of API with different arguments its impossible to implement for each call a detailed implementation. Hence you have to generate analysis functions which are able to handle the arguments more generally. But how to handle void pointers? There is now way to find out how long the pointer value is. Normally an extra function argument exists to define the data length, but as said we have to analyse arguments generally. If an attacker knows the descibed problem he could easly crash the sandbox application and avoid the analysis. As an example a hacker could us the CreateFileA function and handle the string pointer correctly. To avoid crashing the malware program on a real system he uses a invalid file handle. If the sandbox programmer not carefully handle pointers the sandbox application crashes and the attacker attains his goal.
To summarise pointer handling is an important quality measure of sandbox software. I implented a little proof of concept implementation and evaluted each existing sandbox.

• Joebox
Analysis of proof of concept binary
Desciption
• As you see joebox is not vulnerable all API calls have been logged. Befor reading memory joebox checks the memory accessiblity. Furthermore a consequent exeception handling is used.


• CWSandbox
Analysis of proof of concept binary
Desciption
• CWSandbox is crashable (access to noaccessable memory). As consequence the process dwwin.exe was started. The process is used for application error reporting and called as microsoft doctor watson.


• Anubis
Analysis of prof of concept binary
Desciption
• Because Anubis is able to access directly to memory no violation occurses.

Conclusions

Comparing sandboxes with different concepts and different development times is difficult and time-consuming. But it was worthwhile and the results are really surprising. To evaluate precisely each sandbox it is important to include customer needs for each concept. This is however out of scope of this article.
The confidence in virtualisation and emulation software is a very critical aspect for me. It is obvious that a major controll over function interception and code execution exists but many articles have been published to detect these environment implementations. It is well known that malware uses these detection techniques to avoid analysis. Also there are possiblities to subvert these programmes. In addition I do not understand why no sandbox does not use a combination of an analysis machine and a network sniffer machine. Neither Anubis nor CWSandbox are able to analyse driver behaviour. So it is easy to communicate hidden over an own socket driver.
Every sandbox should deal with any kind of pointers with great care. You have seen how easy it is to implement an exploit which crashes the whole analysis environment. Below you find a summary about my personal evaluation of each described techniqual aspect with compared sandboxes.