the myth of AV's failure

i'm sure you've heard it said before that the idea of conventional anti-virus techniques are broken or obsolete, that av is failing... i've touched on the topic before here and here but i think it's time to tackle this myth head on...

people say and believe that conventional av is outdated because of all the reports of it's failure... there are lots of examples where program X failed to stop malware Y from infesting computer Z so could forgive some of the greener minds for concluding that those reports are representative of reality, that anti-virus products don't do a good job...

what anyone with any degree of training in statistics will notice, however, is the obvious selection bias in those reports - you hear about av products failing but you don't hear much in the way of success stories, not because there are no successes but because there's no reason to report them... the failure reports give an entirely lopsided view of the efficacy of conventional anti-virus techniques, leading to the negative perceptual bias the security community now suffers from...

conventional anti-virus (known virus, or known malware scanning) does fail, there's no denying that... specifically it fails to detect new/unknown malware, and that is an accepted and acceptable limitation... acceptable because no single technique can ever be effective against everything (thus necessitating other, complementary techniques) and new/unknown malware, though good for capturing media attention, is not quite as big a deal as it's made out to be... known malware vastly outnumbers new/unknown malware and new/unknown malware (especially that which affects many people, thereby posing a significant threat) generally does not stay new/unknown for long...

now, if known malware scanning really was as broken a model as some people like to think it is, why does a piece of malware's population growth start decreasing once it becomes known malware? it doesn't magically stop trying to spread (or in the case of non-replicating malware, the malware spreader doesn't automatically stop trying to spread it)... the population growth starts decreasing because it begins to fail to spread (or be spread)... the fading out of a particular piece of malware can take years, a decade or more in some cases (usually for reasons unrelated to a scanner's efficacy), all the while it's trying and usually failing to spread... potentially failing orders of magnitude more times than it ever succeeded - and those failures represent successes for known malware scanning...

if people thought seriously for a moment about the life-cycle of malware, about the relative deployment of various anti-malware technologies, and the implications they have for the notion that scanners are failing, falling behind, or just not working they'd realize that in fact scanners must be working far more than they're failing otherwise the population of a given piece of malware wouldn't go down... this is still very much an av world, most people use nothing more than scanners (if they even use that much), there aren't enough deployments of alternative technologies to stem the tide of the malware's spread...

scanning isn't broken or outdated or failing - malware would be a much bigger problem if it were... the notion that it is failing is just an erroneous perception based on an interpretation of incomplete facts...