don't be too proud of this technological terror you've created

lot's of folks have been posting about the 'here you have' mass mailing email worm that's been making the rounds. it's strange that such an old-school technique should inspire so much discussion, but it has and some of it's actually interesting.

one of the discussions comes from the enterprise application whitelisting blog, in other words, it comes from application whitelisting vendor bit9. they are, perhaps understandably, quite bullish about the fact that their technology would have stopped the threat before it could have spread while the anti-virus software vendors were supposedly left to scramble to get detection added after the fact.

while it's true that a classical blacklist or known-malware scanner would require updating after the threat becomes known, it seems that at least some of the anti-virus software vendors that harry sverdlove was taking a shot at were actually able to detect the threat heuristically (see f-secure's post or kaspersky lab's post for example).

it also deserves to be said that many anti-virus software vendors are bundling whitelisting in their suites these days, so people using that feature of those offerings would have been just as safe as if they'd been using bit9's.

most importantly, though - if social engineering can be used to get people to extract malware from a password protected archive sent as an attachment and then run that malware (and we have historical examples of successful email worms that used precisely this technique), social engineering can be used to get people to add that malware to the whitelist.

whitelists do not make you magically immune to this threat. i'm not even convinced they raise the bar a significant amount when you consider how easily people can be tricked into doing all sorts of dumb things. perhaps an enterprise would be in a better position because relatively few would (in theory) have access to modify the whitelist, but administrative users aren't above doing dumb things.

debunking the mythology of whitelist practicality

while i don't normally listen to podcasts, it does occasionally happen and the risky business podcast episode 66 mentioned on the tenable security blog was one of those times... one of the topics discussed was the practicality of whitelists over blacklists and it amazed me (again) that people actually think this way...

why, when the number of developers making good software far outnumbers those making bad software, do people insist on believing there's more bad software than good and it's easier for vendors to keep track of good software than it is to keep track of bad software...

it's a pretty popular belief these days that it's not practical to keep track of all bad software anymore and vendors should be keeping track of the good software instead because that's somehow more practical but that belief starts to look a little ridiculous when you start considering the origins of the good and bad software in the world... just like most people in the world are actually good people (police states would be a necessity otherwise), most programmers are good people too so they're not writing malware... if most of the programmers in the world are writing good software rather than malware then it stands to reason that the production of good software out-paces the production of malware and since it has always been this way it should also outnumber malware...

as such, good software far outnumbers malicious software and is produced at a faster pace than malicious software... however big the set of malicious software seems and however fast it seems to be growing you need to ask yourself how much more aware you are of those stats for malware than for good software (a lot less attention is paid to those figures for good software)... i've mentioned before (and i'll probably mention again) that bit9 actually has some figures related to both the total number and rate of production of good software and it's shocking the degree to which it dwarfs those same measures for malicious software... billions of good programs while there were still less than a million malicious ones, and millions more good programs produced each day while malicious software is still in the range of thousands for the same period...

for the average person this may not seem intuitive; indeed, how could microsoft alone produce 500,000 new files each day - they certainly don't have that many products... the reason for the discrepancy is at least 3-fold... 1) the average person doesn't understand how many different things actually qualify as programs and would need to be kept track of if vendors were to supply whitelists, 2) the average person doesn't realize how many programs go into a single product, and 3) the average person doesn't actually have any idea how many products a company like microsoft actually produces because microsoft produces software for such disparate sets of people... you think ms word is just one program? it's not, it's many different programs that inter-operate to give you the functionality and user experience you're used to... if it were a single program there would be little or no need to install it, you could just run it as a stand alone application... the same holds for excel, and powerpoint, and outlook, and so on and so forth... do you think the hundreds (if not thousands) of megabytes that windows takes up is all because of data? what data does an operating system need? it's mostly programs...

a common refrain these days is that blacklisting just isn't working, but the problem with common notions is that they're often over simplified... blacklisting just isn't working well enough all on it's own... it is a challenge to keep up with the malware production rate so just imagine how much more of a challenge it is to keep up with the good software production rate... sure whitelisting companies like bit9 seem to be able to do it but you wanna know how? by using the same blacklists people think are failing in order to determine what's safe to put on their whitelist... it shouldn't take a rocket scientist to figure out that such a whitelist will be no more accurate than the blacklist it's based on - anything the blacklist misses will get onto the whitelist and then what will you do?

bad really is in the minority

from liam tung's article signature-based antivirus is dead: get over it:

However, there is a problem with the use of blacklists, said Turner. "When the majority of stuff you're handling is malicious, it makes more sense to use a white list because that deals with the exception — blacklists only work if 'bad' is in the minority."

i totally agree with this statement... there's just one thing that turner and just about every other av detractor out there fail to realize... bad really is in the minority... bit9 (an application whitelist vendor) has shown that there are several orders of magnitude more good stuff (on the order of billions) than bad stuff (about a half million at the time) and that microsoft alone produced as many good binaries in a day as there had been bad binaries produced in the previous 20+ years combined (from the bit9 presentation at the international anti-virus testing workshop in 2007)...

furthermore, most of the stuff anyone (other than the anti-malware industry) handles is non-malicious (unless you're looking only at email and are considering spam)... most web pages are safe, most binaries are safe, the majority of stuff most regular people encounter on a day to day basis is safe so if you're going to advocate a security technology that focuses on the exceptions you're going to have to get over your perceptual biases and realize that bad stuff is the exception so blacklists make more sense (at least by that logic)...

you've heard the argument that blacklisting is inferior to whitelisting because the list of all bad things is growing too big too fast, but we have quantifiable proof that the list of all good things is far, far bigger and growing far, far faster... that doesn't mean blacklists don't have serious problems (they do) or that whitelists are unusable (they aren't), it simply means that particular argument is fundamentally flawed and if people took the time to become familiar with the reality of the situation they'd know that...