Identifying system calls invoked by malware using branch trace facilities

We are developing Alkanet, a system call tracer for malware analysis. However, recent malware infects other processes. Others consist of two or more modules or plug-ins. It is difficult to trace these malware because traditional methods focus on threads or processes. Getting the system call invoker by stack tracing is a traditional method to solve this problem. However, if malware has falsified its stack, this method cannot identify it correctly. In this paper, we describe a method for identifying a system call invoker by branch trace facilities. We consider the effectiveness of branch trace facilities for malware analysis.