antivirus testing vs real world occurrences: How can you test what you can’t see?

AV Comparatives March 2012 used 300,000 known samples.  Here testing relied mostly on definitions detection (DAT).  When using known samples, “Brute Force” definition detection is very effective and nets you excellent ratings, with the average at 96% for the top 15 AV vendors. However, this type of testing does NOT mimic real world occurrences.  Polymorphic code is the real problem. All Polymorphic code is unknown, rare, and cannot be analyzed by DAT detection until execution.

AV Comparatives of August 2011 testing is based on static detection of new/unknown malicious software. The average here was just under 60% for the top 13 AV vendors.  The numbers were so low, that AVG, K7, Symantec, McAfee, PC Tools, Trend Micro, Sophos, and Webroot decided to NOT get tested, and therefore to renounce all awards. This is closer to real life scenarios.

This same test model from May 2011 netted an average of less than 50% for the top 12 vendors.  These type of infections are the arch nemesis to all antivirus vendors. The real problem is, how can you test what you don’t know?  How can you test what you have never seen?  Even when testers can identify links to Polymorphic infection engines, they include self defense modules that are “tripped” when more than a single download is attempted to a location. This trip wire ignites a DOS attack using peer Polymorphic engines, and their associated botnets.

I hope this will add clarity to what the real problem is! Polymorphic encrypted code just has to be executed, before any analysis can occur.   With Polymorphism, the programmer, and the program, does not need to know the payload in advance. The exact behavior is only determined at execution. Every sample is unique!

How can we protect ourselves against Polymorphism?

1)           Unplug the Ethernet cable and go offline – This defeats the whole idea of Internet, cloud applications, and email. Can’t we find a happy medium?

2)           Virtualize your browser – This technology is slow to be adopted.  Some functions / websites won’t operate properly.  The browsing experience is slowed, and you can’t save / install the way you normally did. This step offers no protection for email (my Outlook won’t not run in the sandbox).

3)           AutoSandbox 7.3 is the only logical conclusion that I can see.  Even adding a superior antispyware application, like Malwarebytes, or SuperAntiSpyware, does not completely prevent the possibility of infection.



J.R. Guthrie

Leave a Reply

Your email address will not be published. Required fields are marked *