i've got a bone to pick with the av industry (don't look so surprised, i've been critical of them before) that has been stewing for a while now and recent conversations i've had elsewhere have convinced me that it's time to say something...
the problem is the apparent near-exclusive focus on known-malware scanning... don't get me wrong, old malware certainly lingers and known-malware scanning is a quick and easy way to make old malware a non-issue in your environment, but that only works for old malware...
known-malware scanning is not now, nor has it ever been the appropriate tool for dealing with new/unknown malware... for the armchair strategists out there, imagine a situation where you have no intelligence on enemy and then imagine trying to use tactics that require such intelligence in order to be effective... the very notion of using known-malware techniques against unknown malware is an oxymoron - and yet as the set of malware that qualifies as new/unknown at any given time increases, the most obvious reaction from vendors has been to release signatures faster... it's as if they believe they can somehow change the character of the problem of new malware so that known-malware scanning can be the 'solution' to this part of the malware problem too... like they can fix the problem that prevents known-malware scanning from working on unknown malware...
but of course, they don't actually accomplish that... they narrow the window of opportunity a bit for some of the malware, but not enough and certainly not entirely... no matter how quickly signatures are released there's going to be a period of time after the malware's release where it qualifies as unknown and no known-malware technique is going to be suitable for it... furthermore, as the rate of malware production increases the total number of malware samples in that unknown-malware window will increase... while it's certainly necessary to release more signatures per week/month/whatever, getting them out the door faster instead of focusing on more appropriate ways of dealing with the unknown malware problem isn't helping anyone...
instead what it accomplishes is: to cut quality assurance corners (which is the cheapest and easiest way to speed up delivery of anything) resulting in bad signatures that hose customer systems, to make the industry look like a one-trick pony by virtue of the ever escalating effort applied to this single technology, to reinforce in dumb customers the notion that they are staying on top of the problem just by keeping their scanner up to date, and to make smart customers question whether anti-virus has any value at all...
oh, and if you think heuristics are a known-malware scanner's saving grace then you haven't been reading the same retrospective scanner test results i have... the latest retrospective report available from av-comparatives.org (november '07 at the time of writing) shows that by and large the ability of scanners to detect malware using heuristics alone is dismal... only 3 of the 13 products tested managed to score above 50% and even then the best was only 81% (and that product was penalized in the overall rating due to a high false positive rate)... sure 81% is pretty good for heuristics, but it's not good enough to rely on exclusively for protection from new/unknown malware and most were less than half that effective...
av vendors need to start making a much more visible effort in the realm of generic detection techniques, they need to empower people to use those techniques, and they need to stop promulgating this farcical 'solution' snake-oil that makes people think one thing is all you need and instead represent their technologies as tools... that way it's more intuitive that different technologies do different things and that it's often desirable to utilize more than one because there's more than one job to do (more than one problem to deal with)... it makes no intuitive sense to need more than one solution (and believe me when i say people will view the different parts of a security suite as more than one solution) but it makes perfect sense to have a toolbox with more than one tool in it...
4 comments:
Despite working in the AV industry, I've always admitted openly that known-malware scanning is the weakest line of protection against malware. Furthermore, while somewhat effective against viruses, it is utterly useless against Trojan horses.
Viruses spread. They don't reach everybody simultaneously. So, once you see a new virus for the first time, you have the chances to implement protection from it and distribute it to your customers faster than the virus would reach them, thus achieving at least some kind of protection for at least some of them at least some of the time.
Trojan horses, OTOH, are mostly one-shot weapons. With relatively small exceptions, they are rarely re-used. By the time we, the AV people, get a sample of a new Trojan, it has already compromised the systems it was meant to compromise, so implementing detection of it protects basically no-one. (Yes, I know that many in the AV industry disagree with this position of mine.) And nowadays it is the non-replicating malware written for criminal purposes and not the viruses that is doing most of the time. Ergo, known-malware scanning is becoming even more ineffective even faster than before.
(One slight disagreement with you - being a purist, I don't consider heuristics part of the known-malware scanning. A known-malware scanner scans only for known malware and nothing else. The heuristics are used by the heuristic analyzer. While an AV product can - and usually does - include both a known-malware scanner and a heuristic analyzer - and a whole bunch of other kinds of AV programs too - the two are not one and the same thing and shouldn't be compared.)
Unfortunately, the reason why we (the AV people) sell mostly known-malware scanners is not due to some dark conspiracy against the users. The reason is an economical one.
You see, just like everybody else, we have to eat too. So, we're forced to produce what would sell - not what we think would work best. So, we don't make what we think the users need - we make what the users are willing to buy.
And (I know you disagree), 97.24% of any sufficiently large and random group of people are idiots. Including the users. Especially the users. A known-malware scanner is easy to use and gives them a black-and-white answer - like "you don't have a virus" or "you have the XYZ virus, do you want me to remove it?". (The fact that this answer is occasionally wrong is besides the point.)
This is something that the average user can understand. As opposed to that, all other kinds of AV programs five fuzzy answers that the users don't know what to do with. Heuristic analyzers say "this file could contain a new virus". Well, does it or doesn't it? Integrity checkers say "this file was modified". Well, was it infected by a virus or was it modified by the Windows Update? Behavior blockers say "process FooBar wants to communicate with port 123". Well, what the heck does that mean?
The average user is not only stupid - the average user is also ignorant. Furthermore, the average user is not interested in learning about computer security. They just want to be left in peace doing their jobs - i.e., being accountants, doctors, lawyers or surfers of porn sites and not computer security experts. This is why all attempts to educate them have always failed and always will.
This is also why people prefer to buy known-malware scanners. Yes, I don't like it either - but this is the reality. And as long as this is what they are willing to buy, this will be what we (the AV people) will be selling them.
Wake me up when/if that changes - but I'm not holding my breadth.
Regards,
Vesselin
first, i realize that what seems like the useful lifetime of non-replicative malware ought to be over by the time samples get into vendors hands, but i also know that some malware purveyors operate in a sub-optimal manner... whether due to stupidity, laziness, or just being cheap there is still re-use of old malware going on (just as there is re-use of old vulnerabilities)... people distributing the pre-compiled version of the FU stealthkit jamie butler made available at rootkitDOTcom is a prime example...
second, you're right that heuristics isn't strictly a known-malware technique, however nor is it strictly an entirely generic technique... the heuristic rules depend on knowledge the heuristic engine designers gained from malware seen in the past... i didn't mean to suggest that heuristics was a known-malware technique, i only singled it out because it's inextricably bound to most known-malware scanners...
third, i'm aware of the market forces that lead the av industry to where it is today and i can appreciate the fact that there just wasn't much of a market for those alternative technologies... i'm also aware, however, that market forces do not remain constant over time... the fact that there is so much media coverage questioning or outright saying that av can't keep up, is outmoded, is fundamentally broken, etc. suggests to me that we've reached a tipping point where the market (or at least some significant segments within it) may well be willing to accept something new/different...
fourth, i know the kinds of abstract answers that generic techniques have provided in the past but i'm not entirely convinced that those are the only option... it seems to me that the reason the answers are so abstract is because they're looking exclusively at such microscopic details... what if a behaviour blocker, instead of looking at behaviours atomically, kept track of chains of events (ex. process X wrote executable Y to disk and executed it without user input, and now process Y is trying to connect to the internet over port Z)... what if integrity checkers examined the the type of file and it's location when deciding whether to raise an alert and what alert to raise - what if it knew which kinds of files were likely to change and had the ability to make educated guesses from available evidence as to whether the change was legitimate (ex. expect to find some changes in windows files if there's evidence windows update has run since the last check)... i'll freely admit that in the general case keeping track of context perfectly is impossible, but i think generic techniques can acquire and use more contextual information than they have traditionally done...
and finally - the real crux of this is simply that the av industry needs to make their existing generic efforts more visible... many vendors have HIPS offerings now, for example, but in many cases if i hadn't gone and searched for it i wouldn't have known (and in some cases i had to search quite a bit)... most people won't think to even search for it...
My focus here is on home-users
Also let me say, I'm not quite interested in semantic or historical discussions on whether antiviruses are REALLY a broader term than known-malware scanning...
It is true that many antiviruses provide HIPS protection , but if you look at the actual links you posted for Trend, McAfee etc, they are not meant for home-users and are a seperate independent package from the Antivirus. Homeusers will never get those and I doubt many businesses have these either.
Only homeusers who get Kaspersky's, F-Secure's and Panda's (the Internet security suite only but not the AV) really get HIPS...
I have no problems getting such protection from antivirus companies, but the fact is they don't provide it to homeusers!
But in fact there are many many excellent products out there that can be used to fill this gap for homeusers. And a amazing amount are free!
See for example http://wiki.castlecops.com/Lists_of_Freeware_Security_Software
Granted many are crap, but some are not bad, like Threatfire....
@luke:
"It is true that many antiviruses provide HIPS protection , but if you look at the actual links you posted for Trend, McAfee etc, they are not meant for home-users and are a seperate independent package from the Antivirus. Homeusers will never get those and I doubt many businesses have these either."
and that is part of the problem i was trying to underline... sure these companies have generic offerings, but they're invisible to most people... even i wouldn't have known about them if i hadn't specifically gone looking for HIPS offerings from the various vendors... if people don't know about it then they aren't going to use it and they are going to continue seeing the av industry as a one-trick pony...
the av industry isn't doing itself or anyone else any favours by keeping their other technologies low-profile...
Post a Comment