Wednesday, January 14, 2009

virustotal usage FAIL

from rich mogull's post There Are No Trusted Sites: Paris Hilton Edition:
The best part? Only 12 of 37 tested AV vendors catch the trojan. All of who that give me crap for hammering on AV can go away now.


yes, boys and girls - in spite of my prior warning on the matter, in spite of didier stevens' thoughtful post on the matter, and in spite of hispasec's own post on the matter, people still don't get that virustotal is for testing suspected malware not anti-malware...

it doesn't matter if your sample size is 1 or 1000, using bad virustotal results to bolster the argument that av sucks (when it's well known that virustotal's results don't/won't match av user experience) is a big fat FAIL...

rich isn't the only one failing here, though, he's just the most recent example... 'incident handlers' at the internet storm center do this on a regular basis, as do quite a few others...

the devil's in the details folks, start paying attention... since the detective capabilities displayed in the context of virustotal do not represent the real detective capabilities of the products used by virustotal, what point can there really be to posting the detection rates (as dancho danchev likes to call them)? that's right, basically none - not only do they bear no relationship to what is conventionally thought of as detection rates, but also they are NOT accurate...

now repeat after me: virustotal is for testing suspected malware, not anti-malware...

3 comments:

Unknown said...

I think I would normally fail at this also, but your post illustrates a very nice and subtle point about VirusTotal. I get it!

kurt wismer said...

well, if you failed in the past, at least you were in good company...

however, if my post has helped open even one person's eyes then it's a win! now i just have to subtract 1 from infinity a few more times...

Anonymous said...

Well, if we both keep subtracting 1 from infinity, we should get it fixed in No Time. ;-)