350 rub
Journal Highly available systems №2 for 2012 г.
Article in number:
Hybrid approach for shellcode detection
Keywords: shellcode malware botnet detection
Authors:
S.А. Gaivoronski
Abstract:
A common way by which attackers gain control of hosts is through remote exploits. In this paper we consider such exploits (shellcodes) which exploits a buffer overflow vulnerability. In the paper we identify and describe the common and specific shellcode features, and also we propose a shellcode classification which is based on those features. We propose a shellcode detection method that solves the problem of false positives rate minimization while providing the full coverage of detected shellcode classes. Proposed method solves the critical problem of algorithm-s running time minimization as well. The proposed method has been implemented and tested on different data sets. Experiments results show that false positives rate of the proposed method is closed to 0 and efficiency in terms of time complexity is 16-45 times higher than linear combination of existing algorithms depending on the testing data.
Pages: 33-44
References
  1. Wang X., Pan C.C., Liu P., Zhu S. Sigfree: A signature-free buffer overflow attack blocker. In 15th Usenix Security Symposium. July 2006.
  2. Newsome J., Karp B., Song D.Polygraph: automatically generating signatures for polymorphic worms. In: Proceedings of 2005 IEEE Symposium on Security and Privacy (S\&P'05). Washington: IEEE Computer Society, 2005. Р. 226-241.
  3. Wang X., Jhi Y., Zhu S.Protecting Web Services from Remote Exploit Code: A Static Analysis Approach. In Proc. of the 17th international conference on World Wide Web (WWW'08). 2008.
  4. Gamayunov D., Minh Quan N.T., Sakharov F., Toroshchin E. Racewalk: fast instruction frequency analysis and classification for shellcode detection in network flow. In: 2009 European Conference on Computer Network Defense. Milano. Italy. 2009.
  5. Toth T., Kruegel C. Accurate Buffer Overflow Detection via Abstract Payload Execution. In Proc. of the 5th international conference on Recent advances in intrusion detection (RAID'02). 2002.
  6. Polychronakis M., Anagnostakis K.G., Markatos E.P. Network-level polymorphic shellcode detection using emulation. In:Proceedings of the Conference on Detection of Intrusions and Malware & Vulnerability Assessment. Berlin: Springer-Verlag. 2006.
  7. Wang L., Duan H., Li X.Dynamic emulation based modeling and detection of polymorphic shellcode at the network level Science in China Series F // Information Sciences Number 11. V. 51. Р. 1883-1897.
  8. Royal P., Halpin M., Dagon D., Edmonds R., Lee W. PolyUnpack: Automating the Hidden-Code Extraction of Unpack-Executing Malware. In: Computer Security Applications Conference (ACSAC'06). 2006.
  9. Payer U., Lamberger M., Teufl P. Hybrid engine for polymorphic shellcode detection. In: Proceedings of the Conference on Detection of Intrusions and Malware & Vulnerability Assessment (DIMVA-05). Berlin: Springer-Verlag. 2005. Р. 19-31.
  10. Cormen T., Leiserson C., Rivest R., Stein S. Introduction to algorithms, 3rd edition // The MIT Press, Cambridge, Massachusetts. 2009.
  11. Metasploit [HTML] (http://www.metasploit.com/)