imperva's anti-virus study is garbage

Enough is enough! I have had it with these motherf#$%ing flakes on this motherf#$%ing train of thought - (what i imagine samuel l. jackson might say if he were following this nonsense about imperva)

in case you are unfamiliar, imperva (a security vendor of some sort) commissioned a bunch of students from the technion-israel institute of technology to perform an evaluation of the efficacy of anti-virus (all anti-virus as a whole, apparently, rather than comparing them to each other) by uploading 82 found samples to virustotal. yes, you read that right, it's another virustotal-based test.

these days i have a number of alternative avenues to express myself that i didn't have when this blog was still young, and that can often sate my need to express my feelings on some topic. i can make snide comments on twitter, or even parody tweets from a satirical twitter account. in fact i can even make memes about it. unfortunately none of that has proven sufficient in this case because the hits just keep coming.

you see, imperva keeps shopping this quackery out to more and more media outlets where it gets gobbled up and regurgitated uncritically by writers/editors (who really ought to know better if reporting on this sort of topic is part of their actual job) and thus gets put in front of more and more eyeballs of those who realistically can't know better. along the way it can even collect somewhat supporting voices from venerated members of the security community like robert david graham

or wim remes

let me be clear, however - this is all wrong. as has been repeated over and over again, virustotal is for testing samples, not anti-malware. they say it themselves in their about page

The reason is that using VirusTotal for antivirus testing is a bad idea.

and

BAD IDEA: VirusTotal for antivirus/URL scanner testing

those statements alone should be enough but, because virustotal later talks specifically about comparative tests, imperva (and others) have tried to argue that imperva's test is OK because it doesn't compare products to each other. however...

VirusTotal's antivirus engines are commandline versions, so depending on the product, they will not behave exactly the same as the desktop versions: for instance, desktop solutions may use techniques based on behavioural analysis and count with personal firewalls that may decrease entry points and mitigate propagation, etc.

this makes it pretty clear that the product a customer installs is very much a different thing from the program that virustotal uses - they will in most cases behave very differently and so the results that virustotal spits out cannot be considered representative of what actual users of anti-malware products will experience.

(ironically,  a product that appears to fare best in a virustotal-based test may actually be the worst because a higher focus on the type of static (often signature-based) detection that virustotal best measures could be to cover for a weakness in (or absence of) more generic/dynamic detection capabilities.)

but don't just take my word for it, let's hear from a couple of people who actually work at virustotal

yes, that's right, imperva's study is a joke. this shouldn't be surprising to long time readers of this blog since when i first wrote about this problem four years ago the first reason i gave for why you might want to avoid performing virustotal-based tests was that those of us who know better will laugh at you. i'm sure a number of people are laughing at imperva's gross incompetence (hanlon's razor makes me choose this explanation over the more sinister alternatives) but i'm afraid i can't consider the mess they're making to be a laughing matter.

promulgating ignorance in a security context has the potential to do real harm, and that is where i draw the line. that's why i'm writing this, that's why the title gets straight to the point, and that's why i'm going to start naming some names of people/organizations who have helped make this mess and who really ought to have known better. imperva has behaved like a dung beetle, persistently rolling this turd around, but somehow it keeps getting bigger like some katamari damacy of bullshit, and i think it's important to see the scale and scope of it and hold the people responsible accountable. it's worth noting, however, that somewhere deep down someone at imperva must also have seen the potential for their message to do harm - that's why the caveat that they weren't advising eliminating AV was added (as an apparent afterthought).

a non-exhaustive list of people/orgs who really should have known better, tried harder, and ought to be held to account for this growing mess is as follows:

• the technion-israel institute of technology because they presumably performed the test so that the test could have an air of objectivity it would otherwise lack if imperva performed it themselves
• the students who actually did the work because apparently no one bothered to read virustotal's about page
• the students' supervisor(s) who also failed the reading test and whose job it is to look out both for their students and the reputation of the learning institution that employs them
• imperva for coming up with this cockamamie idea in the first place and for later trying to defend it
• rob rachwald who posted the defense of the test on imperva's blog
• tal be'ery who helped with the defense of imperva's test
• amichai shulman (imperva cto) who is quoted far and wide by uncritical reporters
• clinton karr who seems to have written the press release that many others have simply regurgitated as their own reporting, perhaps as a result of getting that press release onto reuters
• the register which took the bait imperva offered
• richard chirgwin who posted an uncritical account of imperva's research
• techworld which took the bait imperva offered
• john e dunn who posted an uncritical account of imperva's research
• cso online which unthinkingly reprinted the techworld article
• the new york times (yes, that's right) who once again took the bait imperva offered
• nicole perlroth who posted an uncritical account of imperva's research
• the wallstreet journal who apparently uncritically summarize the work of others
• michael hickens who ran a summary of nicole perlroth's article as if it was gold inside an aggregate of summarized stories
• dark reading which posted a regurgitation of imperva's press release - is that what dark reading is supposed to be for?
• ihotdesk which took the bait imperva offered
• steven gaskill who posted an uncritical account of imperva's research
• pc magazine which took the bait imperva offered
• damon poeter who posted an uncritical account of imperva's research
• business computing world which took the bait imperva offered
• christian harris who posted a regurgitation of imperva's press release
• security bistro which took the bait imperva offered
• anthony m freed who posted an uncritical account of imperva's research

(i'm aware there's a lot more than this that you can simply find by googling sentences from the press release, i wish i had the time to make this list exhaustive - that said: reuters, the new york times, and the wallstreet journal... that definitely caught a lot of eyeballs)

now, perhaps you're thinking i'm being too hard on the journalists involved here. after all, they aren't experts. frankly, however, they don't have to be experts to see what's wrong with this test. if you're the type of reporter who reports on this type of technology then you should already know about virustotal and about how it can and can't be used. this isn't rocket science, or even some obscure nuance that only matters every 5th wednesday - not in the context of reporting on this subject. this is something reporters covering security technology ought to know. it's table stakes. you need to be this tall to get on the ride.

perhaps you think i'm being too hard on the students and their supervisor(s)? but this is academia we're talking about. they're expected to do their research, and i don't just mean the experimental research, i mean looking up and reading about the issues involved in designing and performing tests on anti-malware products. and their supeverisor(s) should have made sure they were doing their due diligence in this regard. frankly, in my time i've seen lone rank amateurs perform better tests than this with fewer resources. this is not acceptable academic performance.

and as for imperva themselves, well... if you intend to occupy part of the security industry that hopes to steal some of the AV industry's market, then you better know this stuff like it's the back of your hand. the institutional incompetence going all the way up the chain of command to the chief technology officer is astonishing and i'm surprised they managed to find someone with too many dollars and too little sense to give them funding, but i guess p.t. barnum was right about there being one born every minute.

imperva - do yourselves a favour and put a stop to this mess before it gets any bigger. you can't defend this junk computer science, the truth will eventually come out (it seems to have already started). you can't sweep it under the rug either, you've let things get too out of hand. the kind of smear campaign you're currently running was already attempted by the whitelisting industry years before you, and while that industry itself is still around and may even still be pumping out this same kind of junk, it didn't stop them from drifting back into obscurity. the way i see it the only way you can move forward sustainably is

1. admit your error
2. publicly retract your study
3. reach out to the journalists whose reputations have been tarnished by listening to you and apologize
4. assist the students you dragged into this in learning the error of the experimental methodology they followed (you can probably find a lot of good info either on or linked to from the anti-malware testing blog)
5. start over with a more intelligent methodology and try to make your case again with valid data

and if you can't manage to follow these steps then i'll be glad to watch you fade away or get swallowed up in a few years time, because the kind of incompetence you've been proudly displaying so far is not the path to success.

essential FUD

upon reading mike rothman's recent post on categorizing FUD i was struck with a rather surprising realization. not only has the much reviled APT suffered semantic dilution, but apparently so has the seemingly simple concept of FUD.

i say semantic dilution rather than semantic drift because, rather than taking on a new meaning, the apparent elimination of uncertainty and doubt from mike's description means that it's 2/3rds of the way to having no meaning at all. it seems that anything invoking fear is now some kind of FUD - but can that be true? are fear and FUD interchangeable? do we want to make them interchangeable? wouldn't we really only be saving a single keystroke in the process?

i don't agree with mike's characterization of FUD (which seems only fitting as mike doesn't agree with much i say). although i have tried to define FUD before, i've never gone into enough depth that it would contradict interpretations like mike's. that changes today.

if there is one part of fear, uncertainty, and doubt that could clear this all up if it weren't so often completely overlooked - one word that held a surprising amount of meaning - it would be:

AND

it's not fear, uncertainty, OR doubt, boys and girls, it's AND. we're talking about the intersection of the three, not the union. no single one of fear, uncertainty, or doubt qualifies as FUD on it's own. FUD requires the presence of all three. fear is only part of it. i'm tempted to say fear is only the beginning, but that's not true, something as yet unmentioned is the beginning and fear, uncertainty, and doubt are the consequences.

what's going on behind the scenes with FUD, what makes it such a bad thing beyond the simple fact that it's used as a manipulation, is that it introduces an inaccurate mental model that competes with superior ones and the results are rather insidious. mental models inform our actions. they allow us to predict outcomes and consequently allow us to make plans designed to control outcomes in our favour. they are a tool which allows us to effectively formulate strategies for satisfying our basic human needs.

unfortunately, mental models are never 100% complete. there are always holes, always missing pieces and weak points. these are what FUD models exploit in order to compete with existing mental models. obviously if someone's mental model is more complete and internally consistent they are less susceptible to FUD because they "know better" than to fall for it, but unfortunately many people have mental models that are largely incomplete so a FUD mental model has a good chance of taking hold and effectively competing with the model the person had.

that competition is a problem. it causes a person to be confused, to question what they thought they knew. this is the uncertainty - the U in FUD. subsequent to that a person would then logically start to distrust the sources that had informed them and helped them form their previous mental model. this makes it difficult for those or similar/consistent sources to fill in blanks in the original mental model and thus interferes with a person's ability to build a better mental model. this is the doubt - the D in FUD. finally comes the logical conclusion that if what one thought one knew was wrong then the steps one took based on that knowledge could also be wrong. the consequence of that being that the person is no longer prepared or capable of handling something they needed to handle and the emotional reaction to that is fear - the F in FUD.

recapping then, a more in-depth account of FUD is that it is a communicated inaccurate mental model that causes:

• uncertainty about what you know
• doubt in those whom you learned from or could learn from in the future
• fear that you're no longer going to be able to satisfy some need that you have

it should be noted that that same fear can result when you fill in some of the blanks of an incomplete mental model. what differentiates that from FUD, however, is that although fear can result in the short term, there's no uncertainty or doubt. building a better, more complete mental model results in a person being better able develop strategies to satisfy their needs in the long run and is thus beneficial, as compared with FUD which stymies a person's ability to develop effective strategies.

now the argument could be made that mike himself was spreading FUD about FUD (meta-FUD). a model of FUD that seemingly allowed for anything involving fear to be called FUD would certainly make people uncertain about what they previously knew about FUD and doubt the people that had previously informed their opinions about FUD. and since mike also opened the door for the possibility of good FUD and suggested that FUD was more widespread than one would have otherwise thought (as a consequence of dropping 2/3rd's of the requirements), there would certainly be room for people to be concerned that they no longer knew how to navigate the sea of FUD mike was depicting and thus be afraid of getting duped.

on the other hand, however, the argument could also be made that mike's model of FUD is simply incomplete (seemingly missing uncertainty and doubt) and that what might appear to be meta-FUD is actually inaccurate conclusions drawn as a result of missing pieces of that model.

i'm not going to accuse mike of spreading meta-FUD, primarily because i feel accusations of FUD spreading should be reserved for those who should know better than to believe the model they're communicating. those spreading inaccurate or inferior mental models unwittingly should certainly be notified, however.

is secunia the new consumer reports?

(well, looks like i'm going to add to the noise about the secunia test... it's already been discussed on the security fix blog, eset's threatblog, the register, the sunbelt software blog, the panda security blog, and the zero day blog)

so secunia did a test with exploits they developed in the lab and found that av products sucked...

well gee, doesn't that sound an awful lot like the consumer reports test? if you don't make the distinction that exploits are a special case of malware then there would really be no difference between this and that terrible consumer reports test where they paid to have 5000 new pieces of malware created...

but exploit code is a special case, we need to create benign exploits, we need to be able to use them in order to determine whether our systems are vulnerable, whether the patches that supposedly fix the vulnerability have been applied properly, whether they truly fix the vulnerability, etc...

so then this test was alright then, right? nope, not by a long shot... first and foremost is the idea that anti-virus/anti-malware products should detect these lab-grown exploits in the first place... the issue is not so much that av is only in the business of detecting malicious software, it's that there are very good reasons why av can't and shouldn't be detecting benign exploits... as i just got finished saying, we need those exploits, we need to be able to use them, but how are you supposed to do that if your anti-virus is blocking access to them? it's one thing to use a benign exploit to test the vulnerable surface area of your systems, it's another thing altogether to turn off your security software to do so... there are a variety of technical, logistical, and legal reasons why anti-malware must be constrained to detecting only those things with a proven malicious pedigree, and if people don't like that it's just too bad - get over it, those reasons aren't going away just because they don't mesh with your ideology... either exploits are legitimate and necessary, in which case anti-malware apps shouldn't be alarming on them because it interferes with the proper use of exploits, or they aren't, in which case secunia acted in bad faith by creating new malware - secunia can't have their cake and eat it too...

the next problem was this notion of detecting exploitation... read that carefully - "detecting exploitation"... is exploitation a thing? no, it's a behaviour, and despite certain claims from various companies about dynamic behaviour-based heuristics, known-malware scanners (and by all indications that's the only part of the security suites secunia actually tested, begging the question why they bothered with the suites at all - incompetence maybe?) are built to detect bad actors not bad actions... that's not to say anti-malware companies don't have offerings to detect and even block bad or unauthorized behaviour, they do have HIPS offerings, but it's fundamentally different technology from what people are accustomed to with anti-malware and it's not always simple to setup/maintain properly so they don't necessarily bundle it with their anti-malware products or even in their internet security suites...

speaking of the distinction between actors and actions, that confusion seemed to be rooted in the use of the term "threat"... i have in the past remarked that "threat" is a bit of an ambiguous term where all kinds of things with "threat" in their name get called simply threats... in this case in particular, anti-malware apps use the term "threat" as a short form of "threat agent" (which is actually one of the more common things that "threat" is used to represent)... exploitation isn't an agent by any stretch of the imagination but because everything gets called simply a "threat" those who don't really understand what's going on (which surprisingly seems to include the folks at secunia) will treat all usages of the term the same and not realize that anti-malware scanners are only designed to catch some of the things that get called "threat"...

of course, a post on this site wouldn't be complete without pointing out the conflict of interest that is also present in this test... secunia's business is about vulnerabilities and exploits - they have a paid product for detecting vulnerable software (a different approach to the same ends as trying to catch/block the exploits) so it's in their financial best interests to publish a test that makes the anti-malware industry look bad (aka FUD) and the exploit problem look important (in other words, hyping up the problem)... it's a classic self-serving study and one wonders if the people responsible think the rest of us were born yesterday...

the ongoing n.runs saga

you my recall my previous post about n.runs... well, it seems i wasn't the only one who saw FUD as ryan permeh wrote on mcafee's blog about what n.runs was saying specifically about mcafee... now it seems that thierry zoller of n.runs has responded to the mcafee post, or at least he tried to...

he didn't do a particularly good job of it, however, as despite explaining that the graphs come from data gleaned from publicly available 3rd party vulnerability catalogs (something that was clear from their original press release and not in need of additional explanation), he didn't address the issue that ryan raised about not being able to verify n.runs' figures when looking at the raw data and instead mistakenly or intentionally mislead the reader into thinking that ryan was looking to verify the 800 figure (which was n.runs' own) when it was clear from his post that he was only trying to verify the figures that applied directly to mcafee and that were supposed to come have come from 3rd party databases...

thierry also denied making the claim and/or believing that running av makes you less secure in spite of the fact that an n.runs slide deck i posted about last november makes exactly that claim...

additionally, where ryan claimed there was no evidence of these vulnerabilities in mcafee's product being exploited in the wild thierry responds by saying that it's because of the way the vulnerabilities are reported - apparently ignoring the fact that being used in the wild means there should be malware samples implementing the exploit(s) and that mcafee should have seen some of these by now...

one thing that ryan didn't really bring up and so wasn't addressed by thierry is the absurdity of aggregating the vulnerability count across an entire industry (where the 800 vulnerabilities figure is supposed to come from)... it's not an actionable metric, it doesn't say anything about any particular product or vendor within that industry, and only serves to scare people... this is the kind of marketing that john mcafee (long absent from the company bearing his name) used back in the days of the michelangelo virus (have i just invoked the anti-malware industry's version of godwin's law?)... even if there technically are that many vulnerabilities across the product lines of the entire set of vendors in the av industry, it's an entirely pointless measurement...

and while we're on the subject of marketing, am i the only one whose noticed that dancho danchev has put rather a lot of effort into providing a platform for n.runs to spread their marketing message from? one might wonder if he were still as 'independent' as he claims to be, though a more reasonable explanation might be that his rather obvious anti-av leanings (he's frequently made disparaging insinuations on his blog in the past) have been kicked up a notch so that, given the obvious ammo this 800 vulnerabilities claim could provide to an anti-av agenda, he either doesn't care or isn't aware that it's a marketing message he's helping to spread... given a more recent post where he misleads by misusing terms that someone in his position has no legitimate excuse to mix up (samples != variants != families != signatures, so counts of one can't be compared to counts of another), this latter explanation seems all the more plausible...

understanding the malware threat of pirated software

recently both symantec and sophos have come out with statements to the effect that pirated software represents a security risk to users' computers and/or identities...

i haven't seen anyone crying FUD about sophos' claims (at least not this time, maybe next time) but mitchell ashley had some choice words about symantec's claims and, while i think i've expressed myself well enough in the comments to that post, the existence of that post and that sentiment makes me think some explanations need to be made more pronounced than they would otherwise be as simple comments...

the basic premise is that pirated software might damage users' machines or steal their identities... the way this would happen would be that the pirated copy of the software would have some malware type of functionality added to it or perhaps even be completely replaced with malware...

the question is, is that a credible threat and is there a good reason to mention it? if the answer to either of those is no then claims like those made by symantec really would qualify as FUD...

one of the oldest safe-hex tips was to only get software from trusted/official sources... this was to counter the major malware vectors of the day, which were warez (pirated software), bulletin board systems, and floppy disks... i personally have encountered a number of people over the years who ran into problems with malware precisely because they didn't follow this safe-hex principle and while floppy disks and bbses are all but extinct now, the warez scene is still around (otherwise we wouldn't be talking about software piracy) and so is still a viable malware vector... it's not an accidental malware vector either, as numerous virus writers and virus spreaders have in the past demonstrated how rich with computer using risk takers the warez scene is by targeting them and successfully spreading their malware... malware profiteers in today's commercial malware world would have to be short sighted indeed to pass up such fertile and proven ground...

of course, in the specific case being referred to by symantec, the users weren't getting their software from some warez site (at least not to their knowledge)... they weren't engaging in obviously risky behaviour, but rather they were purchasing the software from commercial pirates fraudulently posing as authorized resellers...

now you may think that the fact that they're commercial pirates that are in it for the money obviates the conventional warez-associated risks but let's look at that more closely... just because they're in it for the money doesn't mean they don't have other motives... in fact, if you were a malware profiteer trying to deploy bots (for example), why not devise a social engineering ploy that involved bundling the bots with seemingly legitimate software and enjoy the added bonus that you'd need to charge your victims money in order to make the legitimacy of the software believable?... additionally some of the employees of the commercial pirate organization may be disgruntled enough to tamper with the software or it may get contaminated accidentally - it's not like commercial pirates care about their falsified reputation enough to enable strict quality controls, if something goes wrong they can just blame the actual vendor... speaking of which, the software the commercial pirate is selling may not have come directly from that actual vendor but instead from a warez site with all the associated risks but none of the transparency about those risks that would allow the customer to gauge their risk exposure accurately...

so the threat seems plenty credible to me but still you might wonder whether it's worth it to mention the risk if there haven't been any actual reports of anything bad in the pirated software... if you're thinking like this then i can only tell you that you're thinking like a victim... in my experience victims usually think things are safe unless something specifically says they aren't... if you don't want to be a victim you need to turn that around and (as i'm sure robin bloor, marcus ranum, and many other anti-virus detractors/whitelist enthusiasts would suggest) start considering all software to be inherently untrustworthy by default unless given good reason to think otherwise...

when is a bad test not a bad test?

when it's FUD'ing snake oil...

there have been a number of posts about anti-virus/anti-malware testing recently... even i posted about testing in response to what has become a series of posts about anti-virus testing over on anton chuvakin's blog (1, 2, 3, 4)... well this post is a follow-up because anton has managed to post the original test paper that his series of posts were based on...

to say that i was unimpressed would be an understatement... lets start with the number of samples - you may recall from my previous post that i said that the minimum number samples needed to account for the 2% detection rate that was being claimed was 50... according to the actual paper

Of the 35 malware files, three invalid files were removed from the sample set, leaving 32 malware binaries used in the final tests and performance calculations

so if only 32 samples were used, how is it that the lowest scoring product only detected 2% of the samples? detecting just a single sample gives a detection rate of 3%, not 2%, and all products tested detected more than just one sample... it can't be blamed on anton misremembering the figure he was told either, since the actual test paper states

the lowest was tied between ClamAV and FileAdvisor with a 2% detection rate

in one place and

two products tied for the lowest detection rate at 2%

thankfully the chart with their results clears this up - it's 2 raw detections (not a 2% detection rate) which means a 6% detection rate (which was also correctly reported in that same chart)... now, you'll have to forgive me for calling a spade a spade, but this level of mathematical incompetence (recognizing that you can't have a 2% detection rate with only 32 samples is grade school math and simply reading the column marked "percent" in a chart takes even less skill than that) is inexcusable for people who wish to have their test taken seriously... given such a complete lack of mathematical acumen, it's almost understandable that they failed to realize that a test bed of 32 samples isn't anywhere near large enough to give statistically significant results...

bias also figured heavily in the test... not only because they only used samples that got through the layers of protection already present on the systems they culled the samples from (thereby missing a potentially huge chunk of what's really posing a threat to users and irrevocably compromising the results and the integrity of the test itself), but on a deeper level the test was written from the perspective of an incident response technician... what is the perspective of an incident response technician? well these are the people who spend their days dealing with the after effects of the failure of security software and/or preparing for the next failure so as to make cleaning up after that event easier than cleaning up after the previous one... all they see are the security product's failures because that's their job, and if it weren't for the immutable fact that all security products fail they wouldn't have that job and they'd have to find some other form of employment (such as performing and publishing dubious tests)... just as police (who deal with criminals on a daily basis) are prone to developing an imbalanced view of society if they're not careful, so too are incident response technicians prone to developing an imbalanced view of the efficacy of security products if they aren't exposed to the security product's successes (which are generally invisible by design)... this acute form of perceptual bias taints the entire test at fundamental levels - including the design and methodology of the test (as evidenced by their belief that they need only collect samples that successfully compromised production systems that already had protective measures in place)...

so far these problems would seemingly be attributable to the testers simply being inexperienced and/or suffering from false authority syndrome... it's time to shine a light on a part of the test that can't easily be attributed to that... there was one product in the test that didn't do too badly - in fact it did better than all other products, it detected 50% more than the next best product, and it was one of the few products in the test that weren't used through virustotal... that product was asarium by proventsure (no link for reasons that are about to be made clear)... go and read the paper carefully (it's only 4 pages) and tell me if you can see something a little off about it... yes, that's right - the product that did the best, the product that was better than all others by a wide margin was the product made by the company whose president helped write the paper and is the contact listed in the abstract... the test, which is titled "Antiviral shortcomings with respect to 'real' malware" by gary golomb, jonathan gross, and rich walchuck, is NOT independent... one or more of it's authors has a clear vested interest in making one product look good at the expense of all others... this puts all the other problems with this test into a new and decidedly unfavourable light... the bad math, the sample selection bias, the insignificant sample size, etc. - in light of this revelation they all point to a cooked test designed to make all products other than asarium look worse than they really are (FUD) in order to make asarium look better than it is in comparison to them (snake oil)... the test, therefore, becomes little more than a marketing stunt by a disreputable company whose product should probably be given a wide berth...

and poor anton chuvakin - though widely regarded as a security expert, not only is he clearly not an authority on malware himself but apparently he also can't recognize a fraud/pretender when he sees one... that doesn't bode well for average folks' ability to do the same, does it...

the myth of meaningful informal anti-malware tests

bad tests are not necessarily a problem that is unique to the anti-malware field but it is one that those in the anti-malware community have encountered countless times... it's not that average folks are physically incapable of performing good tests, it's that they simply don't do it...

an informal test lacks strict adherence to established testing protocols, often because the standards set for good tests are so high that most people can't be bothered to go to all that trouble... only those who are dedicated to the process of anti-malware testing have ever done a half decent job of it because it requires a substantial investment of time and effort, not to mention a certain degree of expertise to ensure that the testbed is free of garbage/scanner fodder and duplicates...

i could stop there, i suppose, and hope that what i've said isn't too abstract, but i really object lessons and one sort of fell into my lap... dr. anton chuvakin has used the the results of one such bad test to support the now popular opinion that anti-virus is dead... i saw where this was going early on (the original question was obviously loaded)... i guessed the results would be bad and suggested the most likely reason was that the malware was too new - to which i was told that some samples were weeks old, though weeks can technically still be new enough if no one else has reported it yet... additionally, if a particular piece of malware isn't affecting many people the av companies may down-prioritize it in order to focus on samples that are affecting more people...

the test, as described so far, worked like this: someone who does incident response at a public university collected samples from compromised machines for a period of time and then when the batch was big enough (weeks after the collection was started) this person submitted them to virus total and took note of the results... the results were as follows: the best detection rate was 50%, the worst was 2%, and the average was 33%...

now, i can see a lot of problems with such a test but lets start with the big ones:

1. statistical significance - the sample size was too small... it may sound picky to those who want to believe, but even an unskilled labourer intuitively knows that asking 5 people their opinion on X does not give you a result that can be generalized to the entire population... this test (or at least the interpretation of it that we've been given) pretends to represent the ability of anti-virus products to protect against malware that is currently in the wild affecting people but most likely has a sample size of about 50 (50 is the minimum necessary for a product to get a 2% detection rate, but larger sample sizes make a perfect 50% detection rate increasingly unlikely)... since the wildlist has over 1600 different pieces of malware on it and since that only represents the self-replicating malware (which are believed to be in the minority now), a sample size of 50 (or even 100) just doesn't cut it...
2. sample selection bias - the samples came from compromised machines in a production environment... maybe that sounds reasonable to you but let me ask you this - if you test a variety of lie detectors against people who have proven themselves to be good at fooling lie detectors, are you really measuring how good those lie detectors are in the real world? the answer is no, you're only measuring how good they are against an artificially constructed set of people... the same goes for malware from compromised production machines - the malware you find there is the malware that has proven itself to be good at evading anti-malware scanners, not in-the-wild malware in general...
3. sample selection bias - the samples came solely from computers in an academic institution... sorry to say but universities are not a typical environment... if anything, they are the place where the incidence of truly malicious insider threats are the greatest (uni students practicing their l33t skills)... home users don't intentionally compromise their own computers, enterprise users may consider it (or even try it) but there are more/stronger controls in place (stricter logical access controls, more personally significant administrative controls, etc.) to prevent and/or deter it...
4. test-bed integrity - the test was carried out by incident response personnel... we really have no way of knowing how clean (in the sense of being free of detrimental garbage) the test-bed was... even if it was an incident response technician who was capable of reverse engineering each sample of malware (and had the free time in which to do it), simply reverse engineering the samples would not be enough to determine whether the samples were sufficiently different enough from one another to be considered distinctly different pieces of malware... and since many incident response technicians simply follow a script, there's a serious question as to whether all the samples really were malware instead of scanner fodder and whether any of them were duplicates...

now as i've already said, those of us who have been in the anti-malware community for a while have seen countless examples of these kinds of tests so we know to take them with more than just a single grain of salt... unfortunately the average person doesn't know why they should be wary of such tests and when security experts use these same sorts of tests to prop up FUD that scares people away from anti-virus products it doesn't help anyone... it's bad science and a doctor of physics should know better...

more mobile malware madness

y'know, there are times when i'm not really a big fan of repeating myself (like when it comes to garbage masquerading as insight) but when a supposed top security [/straight face] influencer [straight face] takes a piece of techdirt trash at face value (what's next? will we be citing crackpot, errr, slashdot as an authoritative source?) what's a malware guy to do?

(if this is sounding angry it's because i finally had to unsub from techdirt a little while ago after discovering how inaccurate it is even outside the malware field)

taking the false claims and misrepresentations from the top:

1. f-secure spreads mobile malware FUD - see my previous post on the subject...
2. kaspersky spread FUD about the mobile malware threat - in the original article the kaspersky folks make it clear that you may well not be very likely to see mobile malware in your home country... the dependence on the regional market penetration of susceptible devices (smart phones) on the extent of the risk has been well established..
3. kaspersky took a news crew into a faraday cage in order to make them understand that mobile malware was a real threat - no, they took them in the faraday cage to demonstrate how cabir works...
4. cabir is only a threat when you intentionally ignore the warnings - this is the oft cited yes/no prompt that people consistently think should mitigate the mobile malware threat even though it has been well established that the 'no' button doesn't work (the worm just sends itself again immediately so the prompt comes back as soon as you hit 'no'... if you need to make a phone call your only real options are to hit 'yes' or get out of range of the other infected device because you can't use the phone while the prompt is there)...
5. kaspersky has suggested that mobile malware is common - again, read the original article, you'll find they don't make that claim at all... they do say that there are certain geographic locations where it's more common than others, though...
6. kaspersky could have just found someone with the virus instead of taking the news crew inside a faraday cage - yeah, an anti-virus company is going to demonstrate the functioning of a piece of malware that is epidemiologically equivalent to an airborne biological pathogen without the protective measures necessary to keep it from spreading beyond their control... now pull my other leg...

the myth of overwhelming numbers

one of the myths put forward by the anti-virus is dead crowd is that anti-virus companies just can't keep up with the malware authors... this article about the death of anti-virus, balanced though it may be, contains the following quote that demonstrates just what i'm talking about:

"The traditional, signature-based technologies are simply not able to keep up with the sheer volume of malware that's out there," Jaquith told TechNewsWorld. "There are over 200,000 unique pieces of malware out there. Some host intrusion vendors say that number is closer to a million.

200,000 indeed - that does seem like an awfully big number, how could anti-virus companies possibly keep up? in actuality, this is an example of lying with numbers... although the 200,000 figure is fairly accurate (at least it's in line with what mcafee and f-secure were saying last time i checked), it is not something you keep up with... keeping up is something you do with a rate (a certain number of things per minute or hour or day or year) and 200,000 isn't a rate, it's a total... if we were to turn that total into a rate we'd need to include the time period over which it occurred, and although 200,000 sounds like a pretty big number, 200,000 over the course of 20 years doesn't sound quite so overwhelming...

i could end there, but that would be misleading in the opposite direction... 200,000 over 20 years works out to about 27 per day but that's not the right figure right now, that assumes a constant rate of malware production over those 20 years and it has been anything but constant... in this article stanislav shevchenko (the head of kaspersky labs virus lab) is quoted as saying that november 2006 saw 10,000 malware samples added to their database - that works out to about 333 per day... what's more, it's apparently 5 times the number from january 2006...

now you're probably thinking we're back in the overwhelming numbers range again, but wait - stanislav has more figures, specifically the time it takes for a good analyst to process an average malware sample is 5 minutes... if you add in the stated constraint of a 12 hour shift a good analyst should be able to process 144 samples per day so november's incredible figures could probably have been handled by a grand total of 3 analysts with time to spare assuming there weren't too many samples that were significantly more complicated than average - and even if that assumption is false, anti-virus labs tend to have more than just 3 analysts...

so clearly, right now anti-virus companies are not being overwhelmed by samples... that only leaves the future to worry about... if the numbers increased 5 fold from january to november in 2006, what are they going to be like this time next year or the year after that?...

the first graph shown here depicts the increase in the malware production rate over time for the past several years and admittedly it looks pretty grim but i saw a graph with a nearly identical shape (though obviously with very different numbers) ten or more years ago... somehow the world hasn't ended yet, anti-virus hasn't died yet, and life goes on... the more you do a repetitive task, the more patterns start to emerge and with those patterns comes the potential for more automation which in turn leads to faster malware analysis... advances in automation have sped up and will continue to speed up the analysis process so the future doesn't really look all that bleak either...

[edited to fix the year specified - apparently i'm still getting used to this whole 2007 thing]

anti-virus is not a faulty burglar alarm

wow, what a great analogy robin bloor makes here... too bad it's his reasoning that is faulty...

one of my favourite turns of phrase lately is mismatched expectations - robin bloor is quite clearly suffering from mismatched expectation by likening anti-virus to a faulty burglar alarm... anti-virus is nothing like a burglar alarm, faulty or otherwise, nor is the problem it is trying to solve amenable to a burglar alarm type of approach...

burglar alarms are pretty simple things - you have one or more sensors that detect basic, easily quantifiable environmental conditions (broken window, open door, motion, etc) and an alarm goes off when the sensor is triggered... this is pretty dumb, all things considered, but it works well enough when the thing you're trying to detect can be broken down to such simple events and even better when the home owner can easily decide whether something is a false alarm...

the malware problem, by comparison, cannot be broken down into simple elements quite so easily and end users are largely incapable of deciding whether an alarm from a malware detector is false or not... this is why behavioural detection techniques (which have been around for over a decade) are still not receiving the same kind of mainstream attention that known-virus scanning receives... instead of going the burglar alarm route, anti-virus incorporates a considerable amount of knowledge about the viruses (now more generally, malware) that the vendor has seen in order to minimize false alarms... anti-virus still has false alarms, of course, but just imagine how bad it would be if av were made to be as dumb as a burglar alarm... if we're going to stick to crime-related analogies, anti-virus is really much more like the criminal databases that have proven so useful in the past to keep known felons out of places they're not supposed to go or out of job positions they shouldn't have... not that those databases are perfect, but they sure do help...

of course, bad analogies are not the only thing robin has up his sleeve in that article... he spreads some clever FUD about the malware pandemic using carefully selected figures from a microsoft study i wrote about once before... yes, the malicious software removal tool removed malware from 5.7 million computers, but what robin fails to tell you is that it scanned over 270 million computers... that gives a malware penetration just 2.1% - hardly a pandemic...

all this is to build up to his conclusion that application whitelisting is a superior technology that should be used in place of anti-virus... maybe it is superior, maybe it isn't, it depends on things i've mentioned elsewhere, specifically whether the user can accurately decide what is safe to add to the whitelist and whether the whitelist can cover enough of the various types of program execution (it's not just *.exe's out there)... should whitelisting be used in place of blacklisting (anti-virus)? no, in reality blacklists and whitelists complement each other, they partially mitigate each others weaknesses, so they really ought to be used together...

and just as a point of correction - contrary to mr. bloor's assertion, application whitelisting is not relatively new... the basic idea dates back at least a decade as an anti-virus (gasp!) suite known as thunderbyte anti-virus included a rudimentary form of whitelisting in it's tbcheck module... tbav was a fairly well known product in it's day and there was plenty of opportunity for whitelisting to become popular, but it didn't... others have languished in obscurity too... anti-virus seems to have remained the most popular in part because it required the least amount of knowledge and/or thinking from the end user - and when it comes to malware, that can actually be a very good thing...

malware doesn't hide in search results

the article Malware Now Hides in Search Results has got to have one of the most misleading titles i've seen in a long time...

maybe it's just me but when i saw that i was thinking maybe some clever malware purveyor was gaming search engines to get his/her wares installed on victims' computers - maybe some novel exploit or social engineering trick...

instead its about how there's nothing in the search results... when you use a search engine to search for information on a piece of suspected malware by filename you often won't find anything... yeah, not exactly news to anyone whose been down in the trenches helping people get rid of malware anytime in the last decade or more - i suppose i should have said something sooner so that you wouldn't waste valuable moments of your life reading the article...

but apparently this is something prevx has discovered by analyzing 250,000 filenames... malware is increasingly using deceptive naming tactics - which is true when you consider that virtually all malware uses deceptive naming tactics (when was the last time you encountered properly labeled malware in the wild?) and the set of all malware is increasing in number... supposedly even expert users will find it next to impossible to get malware information this way - though i should hope expert users would already realize how incredibly useless filenames are for identifying malware (or anything else for that matter) what with that expertise including the ability to rename files to anything they choose...

filenames have never been sufficient to identify (and therefore get further information on) a suspected malware sample... what's snowwhite.exe in today's email could be rootkitrevealer.exe in tomorrow's email... that's why known-malware scanners look inside the file instead of just at the filename, and then users use the malware name given by the scanner to look up additional information... the best you can hope for with search results is to use them in a process of elimination sort of way - narrowing down the field of probable suspects by eliminating those which search results suggest are known good files (and then hope that the malware hasn't replaced a legitimate file with itself or actually infected a legitimate file)...

prevx and techworld/cso magazine, in a brilliant portrayal of captain obvious, have essentially just informed masses that you can't judge a book by it's cover...

phishing alarmism

there's a new post over on the securiteam blog that seems to be just a little too concerned about a bank of america suggestion to add an email address to customer's address books... the author seems to believe that following the suggestion will make customers more vulnerable to phishing, that the bank is asking them to lower their defenses...

now, it's not like they're telling people to lower the security settings on their browsers in order to view the bank's website (though perhaps they do that too, i don't know), they're just telling their customers to whitelist them so that their emails don't get rejected by spam filters... the author's contention is that phishers will then start using the same address the customers whitelist as the From: address on their own phishy emails and because the address is whitelisted the customers will be exposed to each and every one of those phishing emails...

but here's the thing, even if bank of america didn't advise their customers to whitelist the email address, the phishers would have used it anyways... phishers posing as bank of america will use any address bank of america uses, regardless of what customers do with that address... there's nothing bank of america can do to stop that and there's nothing customers can do to stop that so it makes little difference whether the customers whitelist it or not... whitelisting the address doesn't mean they can start implicitly trusting email apparently sent from that address, it just means that bank of america's legitimate correspondence won't get lost in the junk mail folder...

of course the phishing emails won't get lost there either, perhaps that's the problem? unfortunately, spam filters aren't really any good at stopping phishing emails... the phishers work hard to make their emails look legit (otherwise they wouldn't fool anyone) so they should be equally affected or equally immune to spam filters as the legitimate bank of america emails are... if the legit emails get through then so will the phishing emails, and if the legit emails don't get through then the users will have to look in their junk mail folders for them and then wind up being exposed to the phishing emails anyways...

basically it's a zero-sum situation as far as phishing goes - nothing the legitimate sender or the receiver do with the From: address can increase or decrease the exposure to phishing so the bank of america customers might as well whitelist bank of america's email address - and security folks might as well wake up to the fact that just because something can be used to a phisher's advantage doesn't mean the user is put at greater risk... spam filtering and anti-phishing are not the same, even though we don't want to see either spam or phishing emails in our inboxes, defenses against one are not necessarily appropriate or applicable towards the other... the officially whitelisted email address just means the phishers don't have to work so hard to figure out what email address to forge on their phishing emails - their convenience does not equal lower security for others...

(and if you're wondering why this isn't a comment on the securiteam blog, it's because i can't leave comments there anymore... they've been consistently rejected as spam for months regardless of content, email address used, or the presence of a url - and the blog admin who's supposed to be alerted to the comment in order to correct the issue if it was misclassified never does, nor is there any obvious way to follow the directions which say to contact him/her about it... they sure know how to make a guy feel welcome...)

anti-virus is dead - not!

well, it looks like yet another (ahem) security expert (albeit one who apparently worked for an anti-virus company in some capacity a decade ago) is sounding the death knell for anti-virus software...

don't be alarmed though, the sky is not falling... people have been prognosticating the end of anti-virus software since the last millennium... it hasn't happened yet and it probably never will..

this example isn't much different than the previous hundred and one instances of av doomsayers... i've written about the supposed failure of anti-virus software in the past - does that address this situation? you betcha... the author (amrit williams) trots out 3 whole examples of worms that anti-virus software didn't stop, but seems to have forgotten the tens of thousands that it did stop... perceptual bias? sure... mike rothman previously made an elegant comment about mismatched expectations and judging by how far outside of av's scope most of mr. williams' examples of av's failures are his expectations are very mismatched...

what he really seems to be getting at is that av sucks because it's not UTM... the argument seems to be that anti-virus on it's own doesn't do as good a job at solving the business problem of security management as a security suite would and perhaps he's right about that narrowly defined part of the malware problem, but certainly not about the malware problem as a whole which neatly transcends your field of vision while you've got business blinders on...

stand alone anti-virus isn't going anywhere, not so long as the world is more than just a collection of businesses, not so long as there are home computers/networks out there, not so long as my security needs differ from your security needs, not so long as best of breed still has benefits, and not so long as we collectively still remember the saying "jack of all trades, master of none"...

stand alone av is evolving mind you, by including detection of additional forms of malware (that most people, and the av products themselves, were calling viruses anyways), but that doesn't stop it from being stand alone anti-virus...

the myth of AV's failure

i'm sure you've heard it said before that the idea of conventional anti-virus techniques are broken or obsolete, that av is failing... i've touched on the topic before here and here but i think it's time to tackle this myth head on...

people say and believe that conventional av is outdated because of all the reports of it's failure... there are lots of examples where program X failed to stop malware Y from infesting computer Z so could forgive some of the greener minds for concluding that those reports are representative of reality, that anti-virus products don't do a good job...

what anyone with any degree of training in statistics will notice, however, is the obvious selection bias in those reports - you hear about av products failing but you don't hear much in the way of success stories, not because there are no successes but because there's no reason to report them... the failure reports give an entirely lopsided view of the efficacy of conventional anti-virus techniques, leading to the negative perceptual bias the security community now suffers from...

conventional anti-virus (known virus, or known malware scanning) does fail, there's no denying that... specifically it fails to detect new/unknown malware, and that is an accepted and acceptable limitation... acceptable because no single technique can ever be effective against everything (thus necessitating other, complementary techniques) and new/unknown malware, though good for capturing media attention, is not quite as big a deal as it's made out to be... known malware vastly outnumbers new/unknown malware and new/unknown malware (especially that which affects many people, thereby posing a significant threat) generally does not stay new/unknown for long...

now, if known malware scanning really was as broken a model as some people like to think it is, why does a piece of malware's population growth start decreasing once it becomes known malware? it doesn't magically stop trying to spread (or in the case of non-replicating malware, the malware spreader doesn't automatically stop trying to spread it)... the population growth starts decreasing because it begins to fail to spread (or be spread)... the fading out of a particular piece of malware can take years, a decade or more in some cases (usually for reasons unrelated to a scanner's efficacy), all the while it's trying and usually failing to spread... potentially failing orders of magnitude more times than it ever succeeded - and those failures represent successes for known malware scanning...

if people thought seriously for a moment about the life-cycle of malware, about the relative deployment of various anti-malware technologies, and the implications they have for the notion that scanners are failing, falling behind, or just not working they'd realize that in fact scanners must be working far more than they're failing otherwise the population of a given piece of malware wouldn't go down... this is still very much an av world, most people use nothing more than scanners (if they even use that much), there aren't enough deployments of alternative technologies to stem the tide of the malware's spread...

scanning isn't broken or outdated or failing - malware would be a much bigger problem if it were... the notion that it is failing is just an erroneous perception based on an interpretation of incomplete facts...

cutting through the mobile malware mess

techdirt, renowned for it's technical acumen (in other words it's signal to noise ratio is just slightly better than the garbage heap of the internet known as slashdot), has a post today that basically roasts f-secure for spreading mobile malware FUD... small problem - i couldn't find the FUD even after following all their links to supposed examples...

let's take a closer look, shall we?

from silicon.com:

Sal Viveros, wireless security evangelist at McAfee, said F-Secure's figures are largely in line with industry figures in terms of the total number of mobile viruses but added such viruses have largely been "proof of concept" to date and pose little threat to users.

ok, so we've got independant verification of f-secure's figures on the total number of mobile malware instances - score 1 for f-secure...

from a different article on silicon.com:

"The number of proof of concept viruses is increasing but that's not to say there has been an increase in the risk of infestation or that there is any need for panic or worry."

the person making this statement (david wood of symbian, the company holding the largest stake in the mobile phone market - aka the microsoft of the mobile phone market) clearly doesn't understand the nature of risk... the more instances of malware out there the greater the chance of a particular user encountering one of them, and therefore the greater the risk...

from the same article:

He added that these viruses will only spread with user permission and conceded that in very rare instances a user could contrive to infect their phone.

which shows that he clearly doesn't understand what's really going on in a mobile infection scenario... the no option doesn't work - you choose no and the prompt just comes back... press no again and the same thing happens... cabir and similar worms will just keep trying and effectively DoS the phone until the user chooses yes... user interaction is a non-issue if the user isn't given a real choice...

(see the video evidence here, it starts about 26 minutes in)

[edit - there's a better view of the video evidence here, starting at about 1 hour and 26 minutes]

from an article at vnunet.com:

"Phone viruses so far have been spreading over Bluetooth, so they only affect phones that are within a few metres. A MMS virus can potentially go global in minutes, just like an email worm," warned F-Secure's antivirus laboratory.

now that is a little troubling that it says minutes - because mikko hypponen, in the video referenced above, says 24 hours (both for mobile phone viruses and for email viruses) and he explains why... it's correct that it has the same potential speed as email worms but minutes seems like an error, either on the f-secure rep's side or (more likely, since they're known for botching these sorts of things) the reporter's side...

at any rate, saying a type of virus has the potential to do X is quite a bit different than saying a particular virus will do X or is likely to do X (which is the implication techdirt makes here)...

and from the a zdnet.co.uk article that triggered the current threat at techdirt:

"F-Secure is saying there's a huge risk of malcode spreading, but they've built this up," said Simon Perry, European vice president of security for CA. "If you look at their behaviour, they've consistently pushed this message. But it's a theoretical, not a real threat," he added.

i don't know where mr. perry is getting this - mikko hypponen (again in the video referenced above) made it seem pretty clear to me that mobile viruses are not anywhere near as problematic as their pc counterparts... susceptible phones are comparatively quite rare, and most of the malware can only spread to other phones that are physically nearby... that doesn't sound like a huge risk to me... he does mention some big total numbers (in the tens of thousands) but considering the law of large numbers as it applies to this situation that doesn't really raise eyebrows...

furthermore, in the same zdnet article an f-secure representative is quoted:

"I have difficulty understanding how this can be bad for [the antivirus] business. This is not a mass problem for all consumers, but our solution is available to those who need it, and there are people who need it today," Impivaara added.

it seems hard to imagine how f-secure could be making mobile malware out to be a huge risk when they're quoted in the media as saying the opposite...

still, techdirt has persisted in laying the FUD spreader charge against f-secure for some time now, not unlike many other community sources (slashdot and digg are the 2 glaring examples) have done to many other vendors... it bears a striking similarity to the reaction you get whenever you suggest there are genuine security risks in mac osx or linux... i thought at first it might just be one site or 2 sites, but the pattern that is emerging seems more widespread - it seems to have something to do with the wisdom of mobs where the wisdom of crowds fails due to the signal to noise ratio being too low... the reality is is that he who yells loudest has the most individual impact on the whole and without sufficient real wisdom to counteract that impact the whole becomes an ignorant mob...

80% of new malware does what to antivirus?

here's the backstory - apparently graham ingram, general manager of the australian computer emergency response team (AusCERT) has revealed that 80% of new malware defeats antivirus software...

now, i haven't seen enough of the full text of his talk to know for sure what he was on about (the media are notorious for twisting perfectly valid statements into horrendously misleading sound-bytes) but saying that the anti-virus products aren't working does not inspire confidence...

in fact, framing it the way he does, talking about new malware and an 80% miss rate is rather misleading too...

let's put things into proper perspective, shall we? when malware is new it is unknown to the anti-malware vendors - that's the nature of things... when it's been around for a while it will no longer be new and no longer be unknown... further, the mainstream anti-virus products are essentially known malware scanners...

now ask yourself, are you at all surprised that known malware scanners don't do a very good job of detecting unknown malware? of course not... now ask yourself, is it really a problem or all that big a deal? no again, known malware scanners aren't supposed to be good at dealing with unknown malware, they aren't meant to deal with that part of the malware problem... unknown malware doesn't stay unknown for long so it will be caught by the anti-virus products eventually, but during that initial window of opportunity you need to employ other techniques and technologies to protect yourself... anti-virus products aren't a panacea, they aren't a cure-all, don't depend on them exclusively but rather practice defense-in-depth - use a multi-layered approach to protection...

the only people who should be seeing a problem here are those naive enough to think that anti-virus products should be all they really need... and doesn't that make you wonder what graham ingram, general manager of AusCERT, was thinking giving quotes that made it sound like the sky was falling?

can joe barr's opinion of the malware industry be trusted?

well, joe barr is at it again... i've blogged about joe once before and where the previous article i wrote about seemed to just be a case of false authority syndrome, this new one about whether the anti-malware industry can be trusted seems to be a more deliberate smear campaign...

when he's not throwing out non-sequiturs like what happened to dan greer formerly of @stake (which isn't really part of the anti-malware industry), he's redressing other non-sequiturs to look like they're actually relevant... for example:

US-Cert knows about the problem of the super-inflated malware numbers in their summary,

except that cert doesn't count malware, they count vulnerabilities - ergo what cert is or isn't doing, what they do or don't know has no bearing on whether the anti-malware industry can be trusted...

then there's innuendo about timing things specifically to make OSX look bad:

The SANS Institute, -- a name which sounds all officious and possibly not profit oriented, but which is owned by the mysterious but definitely for-profit Escal Institute of Technology -- recently did an unusual update to its Top 20 list of vulnerabilities.

They issued their "update" in order to trumpet the assertion that Apple OS X is now just as exposed and vulnerable to malware as Windows. The timing of the release of this unusual "update" is suspicious, coming as it did on the eve of the new advertising campaign by Apple which plays up the fact that Apple is pretty much immune to the types of malware infestations that plague Windows. Previous updates to this list have usually come in the fall: November, 2005; October, 2004; October, 2003; and October, 2002.

what mr.barr fails to acknowledge, however, is that there's a 3rd event in this coincidence - that being the dramatic change in the security landscape of OSX around the same time... 2 viruses and a spate serious vulnerabilities - issuing a report to inform people of the new state of things was a responsible thing for SANS to do...

there's some crazy re-interpreting of that same report, too:

The SANS Institute announcement seemed to be designed to destroy -- or at least bring into question -- the idea that Apple OS X is more secure than Windows. In a document sent to members of the press prior to the teleconference, the SANS Institute wrote:

During the past few months, Apple Safari browser users faced their first zero-day attack. A zero-day attack is one that causes damage to users even before the vendor makes a patch available. In this case, Safari users who just browsed a malicious web site found their computers automatically downloading and executing a malicious file. The user made no error other than to visit the web site. Apple patched Safari to fix this flaw, but almost immediately had to issue a second patch to stop another attack involving email attachments. The experts involved in the 2006 Top 20 Spring update agree that OS/X still remains safer than Windows; but its reputation for offering a bullet-proof alternative to Windows is in tatters. As attackers are increasingly turning their attention to the platform, OS/X vulnerabilities are being discovered at a rapid pace, which could erode this safety in the future.

now, how exactly can the report destroy or bring into question the idea that OSX is more secure than windows when his own quote of the report explicitly says that OSX is still safer than windows? and his later jab (by way of quoting a 3rd party) at the supposed claim that OSX's security reputation is in tatters? the quote clearly shows that the report said OSX's reputation for being bullet-proof was in tatters - which it is... it can't be considered bullet-proof anymore, it's been proven that it's not totally immune to threats...

the real meat of the article doesn't come until the section entitled "From Russia with malice", however... joe barr clearly has a venomous contempt for kaspersky labs, he goes on and on about supposed wrong-doings, such as:

Kaspersky Lab, a Russian Internet security company which operates around the globe, including here in the USA, has been spreading FUD about malware targeting Linux for years. I've cited this example from 2001 before, but here it is again, and it still appears on their Web site. Hey, maybe the SANS Institute used it as a template for their anti-Apple effort. I quote:

Predictions regarding a world epidemic of Linux-viruses have come true in the first quarter of 2001. The latest incidents caused by the Ramen Internet-worm and its numerous modifications, as well as the multi-platform virus Pelf (Lindose) and other Linux-targeted malicious code, have proved that this operating system, (previously considered as the most protected software), has fallen victim to computer viruses.

while one would probably not consider ramen going into the wild to be comparable with the windows worm epidemics like blaster or sasser, compared with other linux malware it was a very big deal... as for pelf, cross-platform infectors have long been considered the means by which self-replicating linux malware would become really widespread and pelf was an indication that such infectors were coming...

of course, since he was chronicling all the perceived misdeeds of kaspersky he had to include 'the case of the non-viral virus' that i de-debunked previously, but then he goes on to describe his disbelief over their linux malware report that showed there were 91 viruses for the linux platform:

I asked Kaspersky Lab if they had any documentation to back up that claim. Jennifer Jewett, a public relations person representing Kaspersky, told me "the documentation sighting the viruses is included in the Encyclopedia on Kaspersky's Viruslist site: http://www.viruslist.com/en/viruses/encyclopedia."

I searched the encyclopedia for Linux viruses and came up with an astounding 972 hits. But just the barest hint of an analysis of those hits reveal that the number would break an industrial-strength bogusity-meter.

strangely, when i did a search for linux viruses on that site, i got 92 hits not 972... just one more than was indicated in the report - most without actual descriptions but at least they include the aliases that other products use so that one can corroborate their existence... is he incapable of using a search engine or just so biased against kaspersky that he can't manage due dilligence? he knew the result set shouldn't have been anywhere near that big, he should have refined his search to narrow it down to just viruses (972 would have been the list of all linux malware, not just viruses, though the number now stands at 976 and will probably change again as time wears on)...

his final bit of evidence against kaspersky came from the recently noted intended macro virus which caused the confusion i wrote about earlier:

After this story was submitted, and the week following another black-eye for Microsoft security in the form of malevolent macros in MS Word, Kaspersky Lab issued another headline-grabbing but bogus alert for a proof-of-concept of the same type of attack on MS Word's largest competitor, OpenOffice.org. Was the timing once more just a coincidence? I don't think so.

since the existence of the malware was independently confirmed and since kaspersky labs didn't create it themselves, the timing was entirely out of their hands... it gets discovered when it gets discovered... should they have kept the first openoffice malware a secret? would it have really served the public to sit on the fact that openoffice is now being targeted by at least one malware writer? somehow that doesn't seem likely...

joe barr concludes that the anti-malware industry cannot be trusted and he attributes all these misdeeds to a desire for more money - so what should we attribute joe barr's misdeeds (ie. his FUD) to?

mcafee-siteadvisor says what about search engine results?

perhaps you've heard - search engines lead users to dangerous content... actually, that's what siteadvisor (a company and technology recently purchased by mcafee) is saying...

siteadvisor, as you may or may not know, provides technology to tell you which of the search engine results on the result page are safe and which ones aren't... they've basically come up with a list of known bad sites and they mark up the search engine results to show you what's bad and what isn't... why they only do this for search engine results i don't know, search engine result pages are not the only pages that link to other pages - it should be possible to mark up all web pages to indicate link safety...

at any rate, singling out search engines this way is FUD... the implication is (and this is born out by all the alarmist revisionist headlines on articles talking about this siteadvisor study) that search engines aren't safe... search engines just point to web pages, just like other web pages point to web pages... it's not search engines that are unsafe it's the web in general - the same malware infested pages you can encounter using a search engine you can also encounter by following links on other pages (which is necessarily true in order for the unsafe pages to get a high enough page rank to be in the first 5 pages of google results)... search engines just point to the rest of the web - their results are an organized reflection of the web pages that exist...

but what is the siteadvisor study really saying? well, that their technology for detecting unsafe results returned from search engines is detecting unsafe results being returned from search engines... thanks, siteadvisor, for something very close to a tautology, and congratulation on validating your raison d'etre... but y'know - it's not unlike when experts on internet addiction do studies that show internet addiction is a growing problem - whether it's true or not, it's completely self-serving...

mcafee-siteadvisor are basically tooting their own horn trying to get attention, and i think i know what kind of attention they're hoping for... you see, back in the day it used to be a pretty big deal to be the anti-virus company that microsoft used in-house... it was a badge of honour, a matter of prestige and great PR... in today's world the internet (rather than the PC) is becoming the new platform - the web represents the file system, the search engine index represents the file allocation table, and the search engine itself is analogous to the OS... how prestigious would it be to be the scanning technology licensed by google or yahoo or msn to clear the bad pages out of their indices? mcafee-siteadvisor doesn't even have to try hard, they're the only game in town, they just have to hype things up a bit and raise some awareness so as to generate market pressure on the search engines to bite (thus the focus on search engines being unsafe)...

3.1% of organic search results and 8.5% of sponsored search results are bad - yup, that should work... not too outrageous, not too easy to ignore, it sounds 'just right'...

FUD from bruce schneier

well, bruce schneier is at it again, spreading FUD about the anti-virus industry colluding with sony-bmg to prevent detection of their stealthkit (what passes for a rootkit these days)... and his kangaroo court readership buy it hook line and sinker, thinking the 'evidence' posted in the original thread was conclusive...

clearly people haven't actually read the original thead for meaning so i'm going to have to dissect the so-called evidence for them...

• f-secure, who's blacklight product was already detecting xcp (the sony/bmg stealthkit), admitted that they didn't immediately alert the public to the (low risk) threat because they knew the script kiddies would jump on the news of a large population of pre-compromized systems (and when the news came out that's exactly what happened) and were in talks with sony when mark russinovich broken the story... it's never been disclosed what those talks were about and it may seem like fun to assume the worst (collusion) but that's just the same paranoia that makes people want to believe anti-virus companies hire virus writers... there are far more reasonable possibilities, such as f-secure following responsible disclosure by trying to convince sony-bmg/first4internet to close the vulnerability in their stealthkit or possibly remove it altogether before releasing information about how the vulnerability could be exploited was released to the public (and scores of script kiddies)...
• norman's sandbox technology was also able to detect xcp before the news broke...
• symantec was implicated in the collusion through a claim that they had approved xcp but that claim was later explicitly corrected (link points to the exact google cache page pointed to in the aforementioned original thread)...
• multiple av companies downplayed the threat that xcp posed... again, one could be a conspiracy nut and believe it's because they were in on it, or one could be reasonable and recognize that it's quite normal for people to downplay the significance of their own failures...
• first4internet (the people who made xcp in the first place) apparently claimed to have worked with the big anti-virus companies to make sure their software was safe, but names were ultimately not given, and first4internet had their own arses to protect so there was plenty of motivation to try to fabricate vague legitimizing circumstances...

the so-called 'evidence' of collusion on the part of the anti-virus companies to avoid detecting xcp, to allow users' machines to become compromized undetectable, is evidence of nothing more than a peculiar desire not to trust anti-virus companies... there is no real evidence of collusion and bruce schneier's continued insistence that there is is nothing more than unmitigated FUD...

bruce schneier who claims to like the "be part of the solution, not part of the problem" metric is sowing fear, uncertainty, and doubt about an entire class of security company when he says things like:

You might have expected your antivirus software to detect Sony's rootkit. After all, that's why you bought it. But initially, the security programs sold by Symantec and others did not detect it, because Sony had asked them not to. You might have thought that the software you bought was working for you, but you would have been wrong.

and that can lead to people away from the only real tools there are for dealing with malware... that's certainly seems more like being part of the problem than part of the solution if you ask me...

of course, when he also says:

McAfee didn't add detection code until Nov. 9, and as of Nov. 15 it doesn't remove the rootkit, only the cloaking device.

he demonstrates that he's provably ignorant of the malware domain... under current parlance, the cloaking device IS the rootkit... under the more classical definition xcp was never a rootkit at all...

the security guru has a blind spot and that's malware, but people accept his word on the subject as security gospel - unable to apply basic logic and available facts to recognize when he's in error, unable to think for themselves... so who owns your opinions? you or some guy whose supposed to know about these things?