virustotal usage FAIL

from rich mogull's post There Are No Trusted Sites: Paris Hilton Edition:

The best part? Only 12 of 37 tested AV vendors catch the trojan. All of who that give me crap for hammering on AV can go away now.

yes, boys and girls - in spite of my prior warning on the matter, in spite of didier stevens' thoughtful post on the matter, and in spite of hispasec's own post on the matter, people still don't get that virustotal is for testing suspected malware not anti-malware...

it doesn't matter if your sample size is 1 or 1000, using bad virustotal results to bolster the argument that av sucks (when it's well known that virustotal's results don't/won't match av user experience) is a big fat FAIL...

rich isn't the only one failing here, though, he's just the most recent example... 'incident handlers' at the internet storm center do this on a regular basis, as do quite a few others...

the devil's in the details folks, start paying attention... since the detective capabilities displayed in the context of virustotal do not represent the real detective capabilities of the products used by virustotal, what point can there really be to posting the detection rates (as dancho danchev likes to call them)? that's right, basically none - not only do they bear no relationship to what is conventionally thought of as detection rates, but also they are NOT accurate...

now repeat after me: virustotal is for testing suspected malware, not anti-malware...

no more non-distribution for virustotal

the hispasec folks announced thursday that they would be getting rid of the virustotal feature that allowed people to scan samples without submitting those samples to anti-malware companies when appropriate...

i think that's great news, because it makes it that much harder for the bad guys to use the feature maliciously... now the hispasec folks may want to believe that wasn't really an issue but lets face it, if there are still script kiddies asking for malware on usenet (and it does happen from time to time) then it stands to reason that some malware writers would misuse virustotal to test their malware... they're people, not perfectly rational automatons, they do things that don't always make sense to us - sometimes because what they know and what we know doesn't always completely overlap, sometimes just because they're cheap and lazy (and aren't we all to some extent?)...

the article also points out that there's a technical reason why using virustotal to test malware with isn't useful for the malware writers - the results don't tell the whole story with regards to the detectability of the malware sample by the various anti-malware products used because virustotal only uses the scanner portion of those products... now, while non-scanner based anti-malware components in anti-malware suites are nothing new, they did fall out of favour for quite a while... thankfully, as the hispasec folks point out, they're coming back and that renders virustotal results meaningless to a malware writer... what they don't point out, however, is that it also renders virustotal results meaningless for the complete tools anti-virus critics who use virustotal results against new malware as proof that anti-malware companies aren't able to stay ahead of the threat...

there are 2 things in the article that baffle me though... first is how anyone could think they're protecting confidential documents by using the non-distribute feature (hello, you're submitting the file to a 3rd party, confidentiality is lost at that moment, not when that 3rd party shares copies with other 3rd parties)... the second is why legitimate anti-malware vendors would need to use virustotal with the non-distribute feature when the bad guys have shown us it's easy enough to setup your own multi-product scanning system...

but all in all i always think it's good when we find ways to be less helpful to the bad guys...