the ongoing n.runs saga

you my recall my previous post about n.runs... well, it seems i wasn't the only one who saw FUD as ryan permeh wrote on mcafee's blog about what n.runs was saying specifically about mcafee... now it seems that thierry zoller of n.runs has responded to the mcafee post, or at least he tried to...

he didn't do a particularly good job of it, however, as despite explaining that the graphs come from data gleaned from publicly available 3rd party vulnerability catalogs (something that was clear from their original press release and not in need of additional explanation), he didn't address the issue that ryan raised about not being able to verify n.runs' figures when looking at the raw data and instead mistakenly or intentionally mislead the reader into thinking that ryan was looking to verify the 800 figure (which was n.runs' own) when it was clear from his post that he was only trying to verify the figures that applied directly to mcafee and that were supposed to come have come from 3rd party databases...

thierry also denied making the claim and/or believing that running av makes you less secure in spite of the fact that an n.runs slide deck i posted about last november makes exactly that claim...

additionally, where ryan claimed there was no evidence of these vulnerabilities in mcafee's product being exploited in the wild thierry responds by saying that it's because of the way the vulnerabilities are reported - apparently ignoring the fact that being used in the wild means there should be malware samples implementing the exploit(s) and that mcafee should have seen some of these by now...

one thing that ryan didn't really bring up and so wasn't addressed by thierry is the absurdity of aggregating the vulnerability count across an entire industry (where the 800 vulnerabilities figure is supposed to come from)... it's not an actionable metric, it doesn't say anything about any particular product or vendor within that industry, and only serves to scare people... this is the kind of marketing that john mcafee (long absent from the company bearing his name) used back in the days of the michelangelo virus (have i just invoked the anti-malware industry's version of godwin's law?)... even if there technically are that many vulnerabilities across the product lines of the entire set of vendors in the av industry, it's an entirely pointless measurement...

and while we're on the subject of marketing, am i the only one whose noticed that dancho danchev has put rather a lot of effort into providing a platform for n.runs to spread their marketing message from? one might wonder if he were still as 'independent' as he claims to be, though a more reasonable explanation might be that his rather obvious anti-av leanings (he's frequently made disparaging insinuations on his blog in the past) have been kicked up a notch so that, given the obvious ammo this 800 vulnerabilities claim could provide to an anti-av agenda, he either doesn't care or isn't aware that it's a marketing message he's helping to spread... given a more recent post where he misleads by misusing terms that someone in his position has no legitimate excuse to mix up (samples != variants != families != signatures, so counts of one can't be compared to counts of another), this latter explanation seems all the more plausible...

defense in depth revisited

so as a result of my previous post on the use of multiple scanners as a supposed form of defense in depth i was pointed towards this set of slides for a presentation by sergio alvarez and thierry zoller at n.runs:

http://www.nruns.com/ps/The_Death_of_AV_Defense_in_Depth-Revisiting_Anti-Virus_Software.pdf

the expectation was that i'd probably agree with it's contents, and some of them i do (ex. some of those vulnerabilities are taking far too long to get fixed) my blog wouldn't be very interesting if all i did was agree with people so thankfully for the reader there's a number of things in the slides i didn't agree with...

the first thing is actually in the filename itself - did anyone catch that death of av reference in there? obviously this is qualified to be specific to a more narrow concept but the frame of reference is still fairly clear...

i'm not going to harp on that too much because that's just a taste of what's in store, actually... the main thrust of the presentation these slides are for seems to be that defense in depth as it's often implemented (ie. with multiple scanners at multiple points in the network) is bad because of all the vulnerabilities in scanners that malware could exploit to do anything from simply bypassing the scanner to actually getting the scanner to execute arbitrary code...

this says to me that instead of recognizing that you can't build defense in depth using multiple instances of essentially the same control, the presenters would rather call the construct a defense in depth failure and blame the failure on the scanners and the people who make them (and make no mistake, there certainly is some room for blame there)... the fact is that it was never defense in depth in the first place and if you want to assign blame, start with the people who think it is defense in depth because they clearly don't understand the concept... in a physical security context, if my only defensive layer is a wall and i decide to add a second wall (and maybe even a third) i add no depth to the defense... an attack that can be stopped by a wall will be stopped by the first one and one that can't be stopped by the first wall probably won't be stopped by subsequent walls...

the slides also have some rather revealing bullet points, such as the one that lists "makes our networks and systems more secure" as a myth... this goes back to the large surface area of potential vulnerabilities; the argument can be made that using such software increases the total number of software vulnerabilities present on a system or in a network - however this is true for each and every piece of software one adds to a system... i've heard this argument used in support of the idea that one shouldn't use any security software and instead rely entirely on system hardening, least privileged usage, etc. but it's no more convincing now than it has been in the past... yes the total number of vulnerabilities increase but there's more to security than raw vulnerability counts... the fact is that although the raw vulnerability count may be higher, the real world risk of something getting through is much lower because of the use of scanners... there aren't legions of malware instances exploiting these scanner vulnerabilities, otherwise we'd have ourselves an optimal malware situation...

another point, and one they repeat (so it must be important) is the paradox that the more you protect yourself the less protected you are... this follows rather directly from the previous point, using multiple scanners is bad because of all the vulnerabilities... the implication, however, is that if using more scanners makes you less secure then using fewer scanners should make you more secure and thus using no scanners would make you most secure... i don't know if that was the intended implication, i'm tempted to give the benefit of the doubt and suggest it wasn't, but the implication remains... again, there's more to security than what they're measuring - they're looking at one part of the change in overall security rather than the net change...

yet another point (well set of points, really) had to do with how av vendors handle vulnerability reports... as i said earlier, some of the vendors are taking far too long but some of the other things they complain about are actually quite reasonable in my eyes and i find myself in agreement with the av vendors... things like not divulging vulnerability details when there is neither a fix nor a work around ready yet (no point in giving out details that only the bad guys can actually act on), condensing bugs when rewriting an area of code (i develop software myself, it makes perfect sense to me that a bunch of related bugs with a single fix would be condensed into one report), fixing bugs silently (above all else, don't help the bad guys), and spamming vulnerability info in order to give credit to researchers (if you're in it for credit or other rewards you'll get no sympathy from me)...

finally, and perhaps the most novice error in the whole presentation was the complaint that scanners shouldn't flag archive files as clean if they're unable to parse them... there is a rather large difference between "no viruses found" and "no viruses present"... scanners do not flag things as clean (at least not unless the vendor is being intellectually dishonest) because a scanner cannot know something is clean - all a scanner can do is flag things that aren't clean and the absence of such flags cannot (and should not, if you know what you're talking about) be interpreted to mean a thing is clean... scanners tell you when they know for sure something is bad, not when they don't know for sure that something isn't bad... if you want the latter behaviour then you want something other than a scanner...

all this being said, though, i did take away one important point from the slides: not only is using multiple scanners not defense in depth, using multiple scanners in the faux defense in depth setup that many people swear by comes with security risks that most would never have considered...