i realize i've been rather quiet as of late - not sure why, perhaps i lost my mojo... anyways, you can all thank cdman for rousing this ogre out of slumber...
in a recent post, cdman lays out his response to randy abrams' post on whitelisting... perhaps it was the hint at the possibility of an ad hominem attack against a fairly well known and long-standing member of the av community (randy was, for a long time, the voice of av from within the belly of the beast - aka microsoft) that piqued my interest, but that wasn't cool so let's move on...
cdman's first substantive beef is the suggestion that whitelisting companies can't do their job without anti-virus software... ignoring the fact that in practice this is actually true (whitelisting companies currently depend on anti-virus software to determine if something is safe to add to their whitelist) lets look at the hypothetical alternatives he suggests - specifically that whitelist vendors could rely on reputation or building the generic malware equivalent of marko helenius' automatic and controled virus code execution system...
relying on reputation offloads the problem of keeping bad software off the whitelist onto the very people providing the bad software... sure people who provide bad software consistently will get a bad reputation and not be trusted, but what about people who only do it once in a blue moon? microsoft releases tons of legitimate and safe software but they have on occasion also distributed virus infected materials... you'd be hard pressed to justify not whitelisting code from microsoft if you were relying on reputation but if you did whitelist all their code you would eventually whitelist something you shouldn't have... furthermore, relying on reputation is precisely the method that customer-generated whitelists are primarily made with, which would make a vendor-generated whitelist using the same technique rather pointless...
next is the idea of building a system to automatically execute samples and perform baseline comparisons to see if the sample compromised the system... and of course this has to be done on a scale sufficient to handle the rate at which sample files are produced (otherwise whitelist vendors wouldn't be able to keep up, much like av vendors supposedly aren't able to)... but have you looked at bit9's (a whitelist vendor) figures? av companies already augment their small armies of malware analysts with automated methods of determining what's bad, and old methods like this are almost certainly among them... if the av vendors can't keep up with the malware then what hope do whitelist vendors have in keeping up with the goodware when it's production rate is (necessarily) several orders of magnitude greater than that of malware? there are all kinds of capabilities peculiar to traditional av companies that whitelist vendors could try to replicate in-house, but the scale of the samples they have to deal with make it impractical for them to do anything other than to replicate the blacklisting capabilities in full in-house and that would mean they would still be using what the general population considers av - it would just be their own...
a third option cdman mentions is using technology like that developed by mandiant... whitelist vendors are unlikely develop such capabilities in-house when it's almost certainly cheaper to buy products/services from others who've already developed those same capabilities, but lets hope in this case they stay away from such ethically questionable companies as mandiant... bad enough that mandiant hires people whose marketability in security is thanks in no small part to their past efforts at making the problem worse, but to then turn around and have some of those same people do essentially the same thing in the company's name at an event like race-to-zero smacks of not just some lapse in HR's judgment but rather of an alignment of moral compasses... perhaps i'm in the minority here, but if a whitelist vendor gets in bed with a company like mandiant i wouldn't touch them with a 10 foot barge pole...
second to the beef about what whitelist vendors would do without av software was cdman's beef with randy's understanding of what actually constitutes a whitelist... i have to admit that my first impression on reading the statement that the TSA implements a whitelist was one of confusion... the most widely known (and reviled) measure the TSA implements is the no-fly list, which is fairly obviously a blacklist... i actually left a comment on randy's original post expressing my confusion but literally as i was writing it it dawned on me that there were other measures implemented by the TSA such as the newly revised rules for flights which basically require one to be granted permission in a 2-stage process before you can fly... of course, as i write this i'm reminded of the various trusted traveler programs that schneier has written about on occasion - those are also whitelists...
despite all the disagreement, though, in the end cdman and randy are actually in agreement about the role of whitelisting - it's simply another layer... both think it's got it's strengths and it's weaknesses, areas where it's more applicable than others, etc.. however, i think randy has once again distilled a complicated topic to a simple analogy when he compares the folks who say whitelists are the end of av with airbags calling seatbelts obsolete... what a clever way to say they're full of hot air...
5 comments:
"We often attribute "understanding" and other cognitive predicates by metaphor and analogy to cars, adding machines, and other artifacts, but nothing is proved by such attributions."
--John Searle
The blogpost in question (randy's, not mine :-)) seemed rather, how should I put it, less logical to me (things like using multiple question marks, etc). On top of that he concludes the post with a metaphor which is not accurate at all.
Regarding the other points: I never said that the proposed solutions would be 100% (nothing is). Also, the biggest problems aren't caused by people who wake up one day and think "I should write a malware", but by people (gangs) who do this for a living.
Also, I used Mandiant as an example. The technology behind red courtain is rather simple and remarkably accurate (I worked on similar systems some time ago which - using machine learning - created a simple ruleset which had ~95% accuracy and ~1% false positive rate). They don't have to be the supplier of the solution, just that they are the only company offering it commercially that I know of.
"The blogpost in question (randy's, not mine :-)) seemed rather, how should I put it, less logical to me (things like using multiple question marks, etc)."
??? you mean like this? is bad capitalization equally as indicative of faulty logic as bad punctuation is supposed to be?
"On top of that he concludes the post with a metaphor which is not accurate at all."
i didn't have any problem with his metaphor... one protective measure does not obviate another... just as airbags don't make seatbelts obsolete, whitelists don't make blacklists obsolete...
"Regarding the other points: I never said that the proposed solutions would be 100% (nothing is)."
they're not just less than 100%... one is a pointless recreation of what customers would do for themselves, and the other is infeasible...
"Also, the biggest problems aren't caused by people who wake up one day and think "I should write a malware", but by people (gangs) who do this for a living."
quantifying the size of problem is like trying to determine how many angels can dance on the head of a pin...
"Also, I used Mandiant as an example."
i know... consider that part of my response as just something i wanted to get off my chest, if you like...
"They don't have to be the supplier of the solution, just that they are the only company offering it commercially that I know of."
all the more reason for me to voice my concerns about whitelist vendors getting into bed with them - whitelist vendors don't have lots of other options...
"??? you mean like this? is bad capitalization equally as indicative of faulty logic as bad punctuation is supposed to be?"
Now that you mention it, it does annoy me somewhat :-P. On a more serious note: usually these marks are associated with postings where the sentiments overshadow logic. Also, IMHO they show a lack of professionalism not acceptable on a corporate blog. It is one thing to rant and rave on a personal blog, and it is an entire different cake to do that on a corporate blog. But this is just my HO.
"i didn't have any problem with his metaphor... one protective measure does not obviate another... "
No, the problem is not there. The problem is that he tries to compare people who advocate a whitelisting-only solution to people who would advocate that you wouldn't need seatbelts. Why the comparison is correct on many levels, it is also meant to induce a "gut-level" response by hinting that whitelisting-only people are somehow ok with people getting killed! Again, we could discuss how and when computer security relates to human lives, but I think that in most of the cases it is much less serious than that.
"they're not just less than 100%... one is a pointless recreation of what customers would do for themselves, and the other is infeasible..."
First of all: a customer might not be well-informed enough to know which are the reputable sources (ok, Microsoft is a no-brainer, but now many people have heard about NirSoft?)
Second of all: no solution is perfect, but (in the case of MS for example) you consider the ratio of clean files versus infected files they released, it is still much better than the miss-rate (false-negative rate) of any AV product.
"quantifying the size of problem is like trying to determine how many angels can dance on the head of a pin..."
I used to work as a professional virus researcher for years, and so I think that I have a pretty good idea of the malware that is out there. The fact is that "amateur" malware is very straight-forward to detect, and also very low volume. The "professionals" are the ones causing the trouble.
"all the more reason for me to voice my concerns about whitelist vendors getting into bed with them - whitelist vendors don't have lots of other options..."
Again, based on personal experience a similar solution could be developed in less than one month. Just (a) take somebody who has seen a lot of malware samples (any (ex) virus researcher would do), make him draw up a bunch of "criteria", take lots of samples (both clean and infected) and throw it at a machine learning algorithm. In just a few iterations you will have a very good classifier.
One more thing (I'm pulling a Steve here): the idea behind explaining something using metaphors / analogy is to concentrate on elements which are very evidently similar between the two elements. The TSA example fails to do this. Of course you can explain into it other things, but it still is a weak (wrong?) analogy.
"The problem is that he tries to compare people who advocate a whitelisting-only solution to people who would advocate that you wouldn't need seatbelts."
technically you are misrepresenting his metaphor... it's not just any old whitelist advocate he's talking about - it's the whitelist advocates who say you don't need av that are comparable to the airbag saying you don't need seatbelts...
"Why the comparison is correct on many levels, it is also meant to induce a "gut-level" response by hinting that whitelisting-only people are somehow ok with people getting killed!"
no offense, but that's something you're bringing to it, it's not actually part of the metaphor...
"First of all: a customer might not be well-informed enough to know which are the reputable sources "
ok, you got me there... a lazy user who doesn't bother to look for recommendations may not develop a reputation system with optimal accuracy... a professional entity will probably do a better job in that case...
"Second of all: no solution is perfect, but (in the case of MS for example) you consider the ratio of clean files versus infected files they released, it is still much better than the miss-rate (false-negative rate) of any AV product."
and part of the reason for the low number of infected releases is . . . microsofts internal efforts to stop malware from slipping into their releases... as i originally said, a reputation based whitelist offloads the work of keeping the bad stuff off the whitelist onto the people actually supplying the bad stuff - which means instead of the whitelist vendor being the av power user, software vendors in general must become av power users (or else risk their good reputation)... this system doesn't remove the need for av in a whitelisting scenario, it just transfers the onus from the whitelist vendor to someone else...
"I used to work as a professional virus researcher for years, and so I think that I have a pretty good idea of the malware that is out there."
you have a pretty good idea of the malware that is being produced, but that's not the same thing as the malware that's out there, nor is it the same thing as quantifying what malware causes the biggest problems...
this can quickly become a semantic debate due to the fact that you haven't qualified the problems you were talking about in any way... i'm not questioning your knowledge or experience - if i'm questioning anything it's your ability to see beyond your own perceptual biases (which is not necessarily an easy thing for anyone to do)...
"Again, based on personal experience a similar solution could be developed in less than one month. Just (a) take somebody who has seen a lot of malware samples (any (ex) virus researcher would do), make him draw up a bunch of "criteria", take lots of samples (both clean and infected) and throw it at a machine learning algorithm. In just a few iterations you will have a very good classifier."
good to hear that the whitelist vendors should be able to reproduce mandiant's capabilities so easily - but we come full circle to the whitelist vendors developing in-house blacklisting capabilities (though not necessarily blacklisting files or byte sequences) and thereby continuing to use av...
"the idea behind explaining something using metaphors / analogy is to concentrate on elements which are very evidently similar between the two elements. The TSA example fails to do this. Of course you can explain into it other things, but it still is a weak (wrong?) analogy."
actually, the analogy wasn't weak or wrong, just unclear... randy has since clarified it in the comments... the tsa creates a temporary whitelist out of the people who pass the screening process (which is a comparison against various blacklists)... with the exception of the temporary nature, this is exactly the same thing whitelist vendors are doing with files...
Post a Comment