polypack isn't quite as bad as i had thought

just a quick update / mea culpa.

although i stand by the general sentiment expressed in my previous post about research not always being victimless, i've finally gotten a chance to look at the specific example of the polypack service (i was unable to before because the site was down and i had to go by what was written about it rather than what was actually on the site).

i don't know if this is a change from how things were previously, but the polypack service is currently not open to the public. that's great news. although it's still possible that some among the select few who do gain access will be untrustworthy, at least it's not a free-for-all, the people behind it did put some thought into the potential consequences - something that's all too rare these days.

it would still be better if they weren't creating new malware at all (why not pack the eicar standard anti-malware test file instead?), but i felt obliged to at least give them credit for not being completely naive about openness.

research isn't always victimless

i read david maynor's negative reaction to an article by roel schouwenberg on the ethics of a particular instance of security research dubbed 'polypack' and, well, i'll be honest - my initial reaction was that someone needed to whack maynor upside the head with a clue-stick.

i didn't run right out and blog about it, mind you. as frustrating as this pervasive lack of understanding is for me (maynor's opinions echo those of other security pundits i've heard countless times before), standing up on a soapbox and declaring 'this is wrong' has not been a particular persuasive course of action.

this has been particularly frustrating to me because these issues are so clear-cut to me and i've been unable to understand what other people's problems are that they can't see it as clearly as i do (or as roel seems to for that matter). to me there is clearly a significant ethical dimension to things such as online services that turn known malware into unknown malware. that said, i'm not the kind of person to shy away from re-examining my own basic assumptions and one of the biggest assumptions i make about other people is that they are, more or less, just like me - they know the same things i know, they think about the same things i think about, they have an awareness of and appreciation for the same things i do. i know, i know, i assume way too much - while it's a nice theory to think that we are more similar than different, there are obviously more differences than i tend to account for. it's difficult to account for those differences since i only really know how i work, not how others do - maybe if i could read minds it would be different.

so, throwing out that assumption i recalled a comment david harley left on a post i made about fred cohen not too long ago. he said he suspected fred didn't really spend all that much time thinking about viruses anymore. it stands to reason that most people, even most security folks (so long as they're outside of the anti-malware subdomain of security) don't actually spend all that much time thinking about malware issues. specifically, i suspect that they don't really spend that much time thinking about the potential consequences of efforts like the polypack project, or the race2zero, or any number of other things i could name. why should they? the consequences are so far removed from them in their ivory research towers (not to mention their lack of focus on the very domain where those consequences would manifest themselves) that they're unlikely to learn of any consequences should they become real.

that's not always the way it goes, of course. look at bootroot, created by derek soeder and ryan permeh of eEye digital security. that was the basis of the mebroot mbr malware that became so well known and prevalent in the wild. the consequences of their actions are really quite plain to see - they armed the bad guys and the bad guys pulled the trigger. they are at least partially responsible for the damages done by this malware - and i say this not as some holier-than-thou security preacher, but rather as someone who has himself quite possibly unwittingly aided the bad guys who made conficker (though i'd like to think it wasn't quite as predictable as the bootroot case). there are real consequences to your words and actions in this field that i think people remain largely ignorant of and i think that if people were more aware of this and had a better appreciation for this then they'd likely see things through the same sort of ethical lens that i do.

i know of one well documented case where this awareness was arrived at a little too late. mike ellison, once known as stormbringer (among other names), was a virus writer in the vx group phalcon/skism who published an open letter in alt.comp.virus in 1994 announcing his retirement from the virus writing scene and recounting his encounter with a victim of his efforts.(from google's archive)

Greetings,
For those of you who know me, you may know my various handles, my activities, and causes for what little fame I may possess... I am Stormbringer of Phalcon/SKISM, Black Wolf, and Jesus of the Trinity, and even some others probably no one would recognize. And today, I am retiring.

Last night, I got email from a guy in Singapore.... a nice guy, really, extraordinarily polite for the circumstances.... he had been infected with one of my viruses, Key Kapture 2, by some guy who wanted to fuck with his computer. It was filling up his drive (he had only 2 megs left) with captured keystrokes, and he had no way to disinfect it, so he wrote me, asking me for help to cure one of my own creations that had attacked his computer... I called him up voice, and talked with him.... and even then he was kind, almost like he didn't really blame me, although I feel he should.
Now, I never released my viruses against the public myself, never wanted them to be in the wild, but it happened. Fortunately, I also never wrote destructive viruses, so I didn't trash the poor guy's computer. It will be fixed, with the main harm being his time and security.

For some reason, this really shook me. I had always written my viruses as educational, research programs for people to learn more from, and for myself to explore my computer and a type of program that I found almost obsessively interesting. All of my viruses were written to explore something new that I had learned, something different, something cool..... and yet, I still managed to hurt someone...

A lot of you people are probably thinking that I'm a wuss, or whatever, and I really could care less.... fucking up people's stuff was never my intention, and yet it happened. I have decided that it is time for me to quit writing viruses, and continue on with something more productive, perhaps even benificial to others.

Don't really know what else to say, 'cept that it was an interesting journey..... and I'll still be hanging around somewhere on the nets.....

Cheers,
Stormbringer, Phalcon/Skism
Ex-Virus Writer

here we have someone who you and i might have considered a bad guy but who didn't consider himself a bad guy. someone who was exploring and researching with computer viruses and when he finally became aware of the consequences his actions had had, when he discovered how real those consequences were, that 'research' came to an abrupt end.

without that awareness, though, it's not so much a matter of ethics for people because they aren't consciously choosing an obviously evil path. saying that they lack ethics is a little bit like calling someone a liar when they don't know what they're saying is false. i'm as guilty of this as anyone, perhaps even more so (just look at how often i've used the ethics tag on this blog). those of us who are aware and appreciative of the consequences refer to it as a matter of ethics because we see it that way, because for us it would be a matter of ethics and we (perhaps incorrectly) think that others should be able to see the ethical dimension as well, should be aware of the consequences of their actions, especially with the not so subtle hint that we think someones ethics are lacking.

hinting isn't working though so let's try some thought exercises instead. for the first exercise i want to you imagine that you've discovered a new way to attack something. now let's say you publish your findings and you put a demonstration up on the web so that people can play with and build upon your creation (for some of you this might be easier to imagine as you've actually done this). and then let's say someone did just that, they built on your work but they did so with malicious intent. now i want you to imagine having a conversation with an ex small business owner whose company went belly up because of the various costs of downtime due to malware based on your designs. do you express regret or remorse? do you feel for this person? how about the mother whose lost the photographic memories of her child because malware you helped create wiped all her data - how would that make you feel? or then there's the retiree who now supplements his diet with pet food because his investments were wiped out with the aid of tools you inadvertently helped create and his pension isn't really enough on it's own to make ends meet. how do you think it would feel to know that you had darkened people's lives?

these are not impossible or even improbable outcomes, but i can imagine that there would still be some reluctance to accept this view. it's very natural - it's called denial. so for the second thought exercise i'm going to stay away from the hypothetical and instead deal with this historical. specifically your history, gentle reader. i want you to remember something. i want you to think back to a point in your past where you hurt someone unintentionally. maybe it was physical, maybe emotional, maybe something else but regardless of the type of hurt it should be something where you really didn't think anything would go wrong. i want you to try and remember it as clearly as possible. remember how easily it seemed to just happen, and remember how hurt the person was. finally, i want you to try and remember how you felt when you realized it was your fault, that it could have been avoided if only you'd done things differently, if only you'd thought about the consequences of your actions. i want you to remember what it was like both before and after you gained that awareness and compare it to how things are for you now - are you really aware of the consequences of your or other people's actions with respect to malware now?

most people should have at least one point in their lives like this that they can think back to. if you don't then you're either some kind of saint, or you lack the self-awareness to realize what an asshole you've been all your life. awareness is a funny thing, though. those of us who have it usually take it for granted, and those of us who don't rarely know it's missing. i'll probably still call it a lack of ethics in the future when people either create or do things that help others create malware, but maybe i'll be more conscious of the possibility that they simply know not what they do, and maybe those who still don't see the ethical dimension to aiding the bad guys will at least have a better idea of where people like me are coming from when we get on their case about it. research or proving a point seem like hollow justifications when victims enter the picture.

(edited: corrected mike ellison's name - thanks graham)

all things come to an end - even endings

not too long ago i published a post called understanding malware growth in which i observed that the period of accelerated malware growth we had seen in previous years appeared to be over. unfortunately, according to new av-test.org statistics (as reported by mcafee) accelerated malware growth is back.

now one way you could take that is that i was wrong. perhaps i was, at least with regards to being optimistic - perhaps optimism was not called for. i wasn't wrong about the rapid expansion being over, though, the growth was more or less constant for the better part of a year. that is a distinct departure from the growth trend prior to august 2007. a year is a considerable amount of time for malware to languish.

another point i don't think i was wrong about was the nature of malware growth itself. if you read the previously mentioned post carefully you should have come away with the impression that, contrary to the naive malware growth forecasts, malware growth is punctuated. there was very clearly a point at which the rapid expansion of malware growth started, and just as clearly there was a point where that expansion came to an end, a point where whatever driving factor was resulting in the higher malware production rates had finally reached it's peak and the malware ecosystem returned to a kind of balance.

i had in mind the possibility that another disruption in malware growth could occur - if it can happen once it can happen again, but i didn't mention it because i didn't want to be negative, i didn't want to jinx our collective good fortune. had more up-to-date data been available, however, i would have known our good fortune had already come to an end when i wrote that post.

another disruptive event does appear to have occurred. continuing with some of the postulates from the previous post (namely that the mechanism for creating the bulk of the variants is server-side polymorphism), if we were to then go with the postulate that the web is the dominant malware delivery vehicle (as opposed to malware being carried directly in email or some other channel) then one of the key limiting factors in the automated creation of new minor variants is the problem getting victims to view malicious content.

there are 2 things in recent memory that i think could have lead to more browsers loading malicious content than before. one of those things is innovation in the field of social engineering - we've been seeing more fake celebrity news (especially deaths) recently than i seem to recall seeing in the past. hammering on a new hook that the public at large hasn't yet grown wise to could result in an increase in the success of social engineering that used that hook and thus trick more people than usual into browsing to malicious content.

the second thing that could be contributing to an increase in browsing malicious content is the increasing trend of mass website compromises through SQL injection. increasing the number of legitimate sites that serve malware (even if indirectly) increases the chances of web surfers stumbling across malicious content entirely by accident.

both of these things can lead to an increase in the browsing of malicious content, which in turn gives the server-side polymorphic engines responsible for handing out that content the opportunity to create that much more of it in an automated fashion. that said, both of these things will also reach a peak in their influence. the public will eventually grow wise enough to the fake celebrity death reports that the success rate of that ploy will drop back down to previous levels. successful sql injection will also eventually taper off as the content management systems being targeted get hardened by their vendors, or as the pool of potential victims shrinks when the entities in it realize they need to take steps to prevent this sort of compromise.

when these things happen malware growth will once again plateau (it might already be there, but it's far too soon to tell) - until the next disruptive malware innovation, that is.

on the efficacy of passphrases

so now that we've all had a good think about whether masking passwords is really the right way to go, let's question another bit of wisdom - let's question whether passphrases are really as secure as we think they are.

you see with all the thinking out loud recently about the password authentication system one of the topics that got discussed in various sectors was the passphrase. the passphrase idea is that instead of using a relatively short string of bytes (which we'll all hope for the sake of argument isn't a dictionary word) as the shared secret you use to authenticate yourself with some system, you use an actual complete sentence which, although made up of dictionary words is quite a bit longer and thus theoretically harder to guess or attack by brute force.

the theory goes that the more characters are in your shared secret (be it a password or passphrase or whatever) the more possible combinations a program would have to go through before it eventually found the one that matched. additionally, since a passphrase is a proper sentence made out of proper words the consensus seems to be that it would be easier to remember.

but are those things true? let's look at the length issue. one of the things you'll often encounter when selecting a password is the need to make it complex. complex in this context means that it draws characters from as wide an array of the printable ascii character set as possible and that it doesn't contain dictionary words - basically the closer it looks like a random string of characters the better. the reason for this is 2-fold: the wide array of characters is to maximize the number of possible values for each character in the password so that the total number of possible passwords a brute force attack would have to go through would be maximized, and random non-dictionary-word property is to remove any constraints on what the various characters might be that would ultimately lower the total number of possible passwords (example. in english the letter Q is almost always followed by the letter U). natural language introduces considerable constraints - although it takes 7 bits per character to represent the entire set of printable characters, the english language is generally known to contain a little over 3 bits of information per character because of all those constraints. that means that an english passphrase of 20 characters is about as strong as a random password about half that length. at least if that were the only issue involved.

now how is your memory? a while back when i discussed choosing good passwords that you should have a different password for each account you make so that if one gets compromised the rest remain secure. nothing about using passphrases obviates the need to have a distinct and different one for each account so can you remember 20, 50, 100 different sentences? can you remember them all perfectly? can you remember which one goes with which site? perhaps not.

how could an average user make that easier (besides reusing passphrases)? the most obvious tactic would be to use culturally significant phrases. things like:

"The rain in Spain falls mainly in the plains"
"Are you smarter than a fifth grader"
"Rudolph the red-nosed reindeer had a very shiny nose"
...

i say this is the most obvious tactic because we've already seen it in the media - if you'll recall way back there was a television program called millenium starring lance hendriksen. hendriksen's character was recruited by a clandestine organization who supplied their recruits with passphrase protected computers, and in hendriksen's case the passphrase was "Soylent Green is people" (a quote from the movie soylent green).

this adds another layer of constraints on the set of probable values and my gut instinct is that this constraint is so significant that rather than mounting a full brute force attack it would be feasible to simply mount a dictionary attack. i will concede that there are a very large number of culturally significant phrases that people might choose from, but i doubt the cardinality of that set differs significantly from that of the set of words in an actual dictionary. furthermore, it wouldn't be too difficult for an attacker to trick people at large into generating a dictionary of passphrases for him/her simply by setting up a free website that requires an account logon in order to access porn.

as such, i question both the idea that passphrases will be easier to remember when used properly and that a passphrase is more computationally expensive to attack. i think the strength gained by increasing the length is a phantom improvement, that the added strength is thrown away due to the various additional constrains on the set of possible values. i think there will be no ease of use benefit as the sheer number of passphrases will still require them to be recorded somehow (and since they're so much longer than normal passwords, recording them all will be more onerous). i like the idea of expanding password fields to accept more characters, but i see no reason to use that extra space for anything other than longer strong passwords.

about face on bruce schneier

well, it was nice to see something as fundamental the issue of password masking questioned (usability expert jakob nielson's original article, bruce schneier's reaction), and then answered (rik ferguson's response, graham cluley's response). i wonder if that indicates we're collectively ready (some of us have been individually ready for a while) to question something equally fundamental like the concept of 'security experts'.

i say this because, as was alluded to by rik ferguson, bruce schneier seemed to be pulling 'blatantly evident factoids' out of his ass (as 'self-appointed authorities' tend to do).

empirical evidence of the decline of shoulder surfing, even if it exists, wouldn't be able to support the implied assertion that it would stay in decline if password masking went away. in fact, the opposite seems much more likely. while it's true that a determined shoulder surfer would be able to figure out your password from your keystrokes, without the password mask even the most casual shoulder surfer would easily be successful. remove the control that makes something rare and it won't be rare anymore.

but i digress. this post isn't about password masking, it's about 'security experts' - and i find myself in a bit of a conundrum because, while i would normally be decrying schneier's posting of material obviously outside his field of expertise, as someone who not only doesn't lay claim to the title of expert but actively rejects it would i not be practicing hypocrisy?

to answer my own question: yes, yes i would. stop that.

the problem isn't (or shouldn't be) that schneier (or anyone else for that matter) posts on topics outside their field of expertise. he should be afforded the freedom to post uninformed opinion just like everybody else. in that regard the problem isn't really bruce schneier at all (even if he is a FUD spreading self-described media whore), the problem, dear reader, is you - for not recognizing how narrow a scope expertise has, where schneier's is, and consequently when to hold his statements up to greater/lesser scrutiny (note: if you have figured out that cryptography is his area of expertise and that you should question everything else then i'm not actually talking to you but everyone else).

and yes, i am aware of the implication that you should then question everything i say - that is actually precisely what i want. one of the benefits from not being an expert that i'd like to enjoy is people not blindly accepting everything i say and actually challenging me when something i say doesn't fit with something they know. i actually like arguing, i find it to have useful properties in the gaining of knowledge, and i think it's a shame that so many seem to value the finding of consensus over the finding of correctness (nevermind the tendency to use authoritative quotes as a replacement for critical thinking). oh well, at least nick fitzgerald and vesselin bontchev were still willing to expound at length on the topic of malware last i checked (though it has been a while).

month of 'do as i say'

well today is the first day of july and that means it's also the first day of the month of twitter bugs. like past 'month of X bugs', each day a security vulnerability relating to the subject product/service (in this case twitter) will be disclosed to the public.

why are they disclosing these vulnerabilities to the public? as with any exercise in full disclosure the idea is to force the company(ies) involved to clean up their act and fix the disclosed vulnerability. full disclosure puts the information into the public sphere so that many people are aware of it and something as attention-grabbing as a 'month of X bugs' captures even more mind-share and gets it that much closer to the mainstream.

in theory this is good for the public. in theory the researchers doing the disclosure are doing a good deed because they care about security and the company(ies) they're pantsing are lazy, arrogant boors who don't care about security or the public and who need to be forced into doing the right thing. and supposedly fixing the individual bugs the vulnerability researchers identify is the right thing to do.

but what if the theory is wrong? not all companies are created equal (nor are all vulnerability researchers for that matter). nobody knows what goes on inside a company except the people inside that company. it's possible that a given company may actually be working on a comprehensive upgrade to their security posture that would obviate swaths of bugs including the ones that get identified by outside researchers. but that would have to be delayed and the company's time wasted so that they can chase down these individual bugs one at a time in order to appear to be doing something. such is the consequence of shifting vulnerability notification to the public sphere.

now, i'm not going to be disingenuous and suggest this is what happens all the time or even most of the time, but being in software development i feel confident that it is happening some of the time.

external full disclosure practitioners have no way to know that, however, nor have i seen any indication that they care. despite the warm and fuzzy ideology behind the practice, full disclosure is an application of force. it is an exercise in coercion, an example of using public relations to bend a company to the vulnerability researcher's own will because supposedly the researcher knows better than the company what is best for the company's users.

i'm no more a fan of lazy, arrogant, boorish companies than the next guy, but belligerent 3rd parties leave a bad taste in my mouth too.

live by the sword, die by the sword

so yesterday i got a rather rude surprise by email. google, in their infinite wisdom, had decided to label this blog as spam and had locked it, preventing me from publishing what i had planned to publish. if i hadn't acted within 20 days the blog would have been deleted, apparently.

yes, it was the real google and not a spear phishing campaign (it would have made for good social engineering but who'd spear phish me?). they used the exact email address that i only gave to google for my blogger account, and furthermore, when i visited my blogger dashboard (not following the link in the email) it showed the warning about it there plain as day so it was definitely for real.

according to the literature i was seeing, the system by which they identify splogs is automated. they mentioned fuzzy logic, but we know what that means - heuristics (though not the same heuristics used in anti-malware, obviously). although i'm not exactly some hardcore heuristic fanboy i can't help but feel there's a bit of irony in me (or more specifically this particular blog) catching the business end of a heuristic false positive.

there is, of course, an appeal process they give you (so that your blog doesn't get deleted after the 20 day grace period) but i found it a little annoying that there was no indication anywhere that i'd successfully initiated that process and it was only when i checked again today and found the splog notification gone that i had any clue that anything had gone through. the real test, of course, is publishing and that's part of what this post is meant to achieve - if you're seeing this then the blog is back to normal.

understanding malware growth

(this is actually kind of old now, but real life has a way of interrupting this whole blogging thing)

alex eckelberry posted some graphs depicting the size and growth of the malware samples collected by andreas marx/av-test.org. what most people will take away from it is that malware is growing at an incredible rate, and that's kind of depressing. there's more to the graphs than just that however, and not all of it is depressing - in fact it looks pretty good to me.

but to understand this i think i need to point out just how bad the forecast line of the graphs is. the line represents the expected value according to a particular model of malware growth. of course models and reality never match up absolutely perfectly, but the forecast suggests a fairly simplistic model and it appears that as time wears on it diverges more and more from reality. let's construct a model bit by bit based on some reasonable assumptions and see how close it matches reality.

let's first suppose that there are a constant number of malware creators with a fixed amount of resources (i don't mean money or computing power, i mean man-power resources - you can only do so much in a day, that sort of thing). let's further suppose that the amount of malware a malware creator can create with X number of resources is constant. what would that look like?



this depicts constant growth with a linear increase in population over time. this is not the final model, of course, but simply a starting point. one of the basic flaws with it is that it assumes that the population of malware creators themselves is constant. what if we instead suppose that the set of malware creators themselves were growing at a gradual but constant rate, what would that look like?



this depicts linear (rather than constant) growth in the malware because as the set of malware creators increases over time, so too should their malware output. it also shows a curve in the population vs time graph, although the curve isn't quite as pronounced as the one depicted in the forecast on the population graph on alex's blog and that's because in that case the forecast on the growth graph predicted non-linear growth.

how could we get non-linear growth? well another of our starting assumptions that we need modify is that the number of malware samples a malware creator can create with X number of resources is fixed. in reality malware creators come up with innovations and some of those innovations improve the efficiency of their malware creation efforts such that they can create more samples with the same effort as before. minor innovations in this area (minor in the sense that the efficiency improvement is small) probably happen all the time but major innovations are probably rare. furthermore, adoption of such innovations is not immediate, it takes time for the idea to spread amongst the malware creators. finally, such innovations may not apply to all sorts of malware so it may be that only part of the malware creator population ever adopts the new methods.

so now let's suppose one of those rare groundbreaking innovations happens - one malware creator comes up with a way to increase his malware creation rate by an order of magnitude. then he tells 2 friends (it may not actually work quite that way but one way or another the idea gets communicated to others) and they tell 2 friends, and so on until eventually they approach a peak number of creators using the new method and the rate at which creators switch from the old to new method slows to a trickle. what might that look like?



this depicts the same linear growth as before at the beginning (with the scale adjusted) followed by a period of rapid expansion as the new method reaches it's saturation point amongst the malware creators (most, but not all of them adopt the new method), and then it returns to mostly linear growth but with a stepped sort of appearance as individual adopters of the new method gradually continue to trickle in.

now, i don't know about you but to me this is starting to look rather familiar. it's not quite the same shape as the actual figures, mind you, that looks a lot less smooth than this, but it's a better approximation to the shape than the simple curve used in the forecast and it's based on an entirely reasonable set of forces that would affect malware growth.

as a final step, let's try to imagine why the real values don't follow such a smooth curve. so far we've been assuming that the population of malware creators only increases over time, but that's not entirely true, is it. there are any number of reasons why individuals might leave that population either temporarily (due to things like criminal prosecution, or collapse of particular infrastructure they were using), or permanently (due to things like retirement or death). if we were to take some of the malware creators in our model out of the picture temporarily at various points (and assume that the ones that ones that leave permanently will simply be replaced and so have the same effect as a temporary departure) what might that look like?



as you can imagine, because the malware creators using the new methods contribute so much more to the growth of malware than do the ones using old methods, when the population using the new methods drops slightly it has a much more noticeable effect on the graph than when the population using the old methods drop.

so how reasonable are these assumptions? are malware creators resources fixed? i've yet to see a way to increase the number of hours in a day so any individual malware creator is still limited in the number of man-hours he or she can devote to the task of malware creation.

can an innovation have a huge effect on the number of malware samples created? well just look at the massive number of variants that server-side polymorphism has lead to. you can basically think of those setups as variant factories and once they're constructed they really don't require any extra effort on the part of the malware creator in order to create more samples - it can be entirely automated and is then only limited by the number of victims who download samples.

would adoption of such an incredible innovation really be limited? again, look at server-side polymorphism, how well would that innovation apply to malware such as autorun worms? not that great actually. if you think about it you can probably come up with other examples of malware types and malware techniques that don't naturally go well together.

do people really leave the malware creator population either temporarily or permanently? well first of all people do actually die, and malware creators are among them so yes people do leave permanently - and again, with the example of server-side polymorphism, if the servers go down because for example the hosting ISP gets de-peered (as has famously happened a few times now) then yes the creators will temporarily be removed from the malware creator population until such time as they can set their operation back up somewhere else.

so what else can we take away from those graphs at alex's blog besides the fact that malware is increasing at an alarming rate? well, for my money it's that a big chunk of that alarming rate was due to an innovation (probably server-side polymorphism) that created a rapid expansion in the malware creation rate and that that rapid expansion seems to be over (in fact it looks like it came to an end nearly 2 years ago). that's not to say malware growth will begin to slow of course, but simply that the really scary part is behind us - the malware growth rate is still enormous, but it's not increasing at an exponential rate anymore. maybe a new innovation will come along and take us for another rough ride, maybe not, but as it stands right now catching back up to the bad guys (assuming you were of the school of thought that we needed to) looks a lot more feasible than it did before.

now read this!

there was a while there where i was posting links to other peoples blogs to help drive traffic to what i thought were good posts. i fell out of that habit, however, as it became forced and formulaic.

that doesn't mean i don't still come across good posts, but good just isn't really good enough - the criteria of "good" is too nebulous, it gets watered down and loses all meaning.

so now the fact that i'm pointing out a good post again actually means something because it's not just good - it's good enough that i felt moved to do this:

richard bejtlich's post on obama's cybersecurity speech (despite our philosophical disagreement over the classification of malware as a threat) was an excellent bit of creative re-writing of the original speech. for a moment i almost thought i was reading sun tzu again (yes, i am one of those people who reads and re-reads the art of war). if only such insight could have really come from the top like that (or at least been repeated by the guy at the top for everyone to hear).

fred cohen says anti-virus doesn't work

the EICAR (european institute for computer antivirus research) conference was held earlier this week and something which may at first blush seem quite curious happened... fred cohen (a keynote speaker, the father of computer viruses, and the man behind the seminal academic treatment of the subject) said that anti-virus doesn't work...

i wish i'd been there to see that (actually no i don't, i almost certainly would have rolled my eyes, crossed my arms and gone to sleep), but since i'm not part of the industry and flying over to berlin on my own dime is kinda out of the question i had to miss out... it seems i wouldn't have even heard about it if not for randy abrams posting about it here... i've waited patiently for a couple of days now for anyone else to write something about it so i could get more details of what was actually said, but that doesn't seem to be forthcoming... even eicar's site only tells me that the title of his talk was supposed to be "Computer Virology: 25 Years From Now"...

so, since i don't have first hand experience of the conference to give me a clue as to what on earth he was thinking, i went and did some research to see if anything he's said or written in the past might give me a clue and i believe i've found what i was looking for...

in an article on his site titled "Unintended Consequences" it starts to become clear that he feels the computer security industry is in the practice of detecting and reacting to bad things instead of preventing and deterring them, and he paints that in a rather negative light... when i read this i was struck - if he thinks anti-virus (probably one of the most mainstream parts of the computer security industry) is about detection and not prevention then he's doing it wrong!... in my earlier post on defensive lines in anti-malware i illustrate pretty clearly how anti-virus technologies (both the ones that security numpties consider av and the ones they don't) are employed as preventative measures, preventing such things as access to threats, execution of threats, and behaviour of threats (among other things)...

the thing is, fred cohen is really not the sort of person you'd expect to be using av wrong... thankfully i found a pair of interviews with him - one called "Three Minutes With Fred Cohen, Virus Trends Tracker", and the other "Fred Cohen - Not all virus writing is a crime"... although anti-virus is certainly used for prevention, it's underlying mechanisms are largely detection-based - however when cohen talks of prevention he's referring to the type where the underlying mechanism is sort of like an immunity... it all became clear to me when he said we should be using systems that are less susceptible to viruses in the first place - systems that have more limited functionality... this relates back to his 1984 paper Computer Viruses - Theory and Experiments where, in the section on prevention (i really should have looked there earlier) he discusses the 3 key properties of a system that allow viruses to spread - sharing, transitivity of information flow, and the generality of interpretation...

systems with limited functionality refers specifically to the third of these properties - the generality of interpretation - by limiting the functions that a system can perform you limit what can be done as a result of interpreting data as code... cohen poses the examples of turning off macros in ms word, or turning off javascript in the browser - i would follow those up with more contemporary examples such as microsoft recently announcing the disabling of autorun for flash media, or the increasing trend (at least among security conscious users) to use alternate PDF viewers as opposed to Adobe's own Acrobat Reader which has been bloated with unnecessary (and frankly unsafe) functionality for a long time...

i'm not going to try and detract from his arguement by saying the idea doesn't have merit - it does... i even considered whether my defensive lines post needed another line regarding system immunity, though i later realized that limited functionality is already covered by existing lines (as it's just another method of preventing exection and/or particular behaviours)... the problem isn't that his argument in favour of using this technique has merit, the problem is figuring out how he could think anti-virus is broken and this technique isn't - or what he thinks the security industry can do about limiting functionality...

the very existence of the security industry underscores the fact that there has historically been no (and largely still is no) place for security in the rest of the computer industry... limiting functionality is the domain of the original product developers, not 3rd party security companies - the norton's and mcafee's of the world don't have the freedom to cripple Acrobat Reader for example (at least not without getting sued)... and the original product developers by and large aren't limiting functionality because that goes against consumer expectations for technology... technology is supposed to empower us to do more, to do things better/faster, etc - so it's a given that when we choose something we expect to empower us, we aren't going to go with the less powerful option... this is born out by history as well because there have always been less functional/powerful options available and the market has consistently selected against them in favour of the bloated product suites and application platforms...

furthermore, limiting applications is all well and good but there's plenty of malware out there that exists as native binary executables, and limiting the operating system's functionality (or worse, that of the hardware itself) to prevent things like self-replication or botnet command and control communication is a far trickier proposition...

so given the difficulty in getting more adoption of limited systems and given the fact that even cohen isn't suggesting limiting things to the point of systems with fixed first order functionality, i can't help but wonder how he could consider this any less flawed an approach in practice than anti-virus... it is clear to me that it will be just as possible to work around the limitations he's proposing as it is to work around anti-virus technologies, and limited functionality defenses are a lot less agile than things like known-malware scanning so when those special cases come along it will cost a great deal more to redesign a system to further limit functionality so as to deal with such special cases...

that should give you a clue as to my opinion of limited functionality-based defenses - they have promise as a complement to anti-virus techniques... anti-virus may not work in the sense that it's not completely effect at preventing malware infestation, but limited functionality systems aren't either... anti-virus may not work in the sense that it's a computationally expensive approach to prevention, but limited functionality has logistical costs for development, marketing, and maintenance that we may never completely overcome... i fully endorse the idea of selecting software that only has as much power and functionality as you need and no more, but i still think you should use anti-virus right along side that...

frankly, at the end of the day, the main difference between systems that are designed to have more limited functionality and ones where anti-virus is deployed is that the former's limitations are baked in while the latter's limitations are bolted on (because like it or not when an anti-virus stops a piece of malware from executing on your system it is effectively limiting your system's ability to perform the function that malware represents)... that appears to be what cohen's real contention is about - that baked in is better than bolted on... in some ways that may be true, but in others (like the matter of agility) it's definitely not (because it's easier to change the bolt than it is to change the whole system)...

defensive lines in anti-malware revisited

now, i realize i've covered the topic of defensive lines in anti-malware before, about 2 years ago, but i've refined my thinking since then and now i've found the motivation to share (guess i was just being lazy before)... this thread on wilders security forums made me realize that there are some misconceptions about defensive lines in anti-malware... specifically the ideas of what can be the first or last line of defense seem to require some refinement so here goes...

if you're thinking about multiple lines of defense then it makes sense to consider that there is a sequence of defensive lines at various 'distances' from the final outcome of complete system compromise... that is to say that there are various opportunities (some earlier, some later) during the course of a piece of malware's progression towards compromising a system at which you can prevent a system from becoming compromised, and each of these points a defense can be mounted... the first line of defense, therefore, should be the one furthest away (or at the earliest opportunity) from that possible outcome... likewise the last line of defense necessarily must be the last chance for preventing that outcome...

so what is the first possible line of defense then? well, first opportunity you have for avoiding having your computer get infested with malware is the point of initial exposure - if you don't encounter the malware in the first place then there's nothing else to say about it... when it comes to defending oneself the main defense here is actually risk aversion behaviours like not going to so-called 'dodgy' sites, not plugging strange flash drives you found on the ground into your computer, etc... when it comes to defending others, any malicious site take-down effort will also help prevent people from coming across the malware on malicious sites that get taken down... somewhere in the middle are tools like mcafee's site advisor, or google's malware warning functionality that can help users determine which sites to avoid...

the next logical point at which you can prevent compromise is preventing the transfer of the malware onto the machine we're trying to protect... for automatic transfers (such as drive-by downloads or vulnerability exploiting worms) the primary defense is closing the avenues by which such automated transfer can take place, usually by applying security patches (at least where browser exploits are concerned)... a basic NAT enabled router can also protect machines behind it from certain types of automatic transfer by virtue of preventing unsolicited traffic to those machines... for manual transfers, risk aversion behaviours again play an important role - only downloading software from reputable sites (preferably direct from the vendor's site) being a prime example... a somewhat after-the-fact but related defense is on-demand scanning all incoming materials (whether you think they could carry malware or not) before allowing them to be accessed for any other purpose (sort of like a mandatory quarantine period)...

after potential malware has successfully been transferred onto the system the next opportunity to prevent it's progression to full compromise is preventing access to it... this is where on-access scanners come into play... if the user can't access the malware then there should be no way they can trigger it...

if your malware access controls fail to prevent access (say because the malware is packaged within some obfuscating wrapper, or perhaps the malware is just new) then the following point at which full compromise can be avoided is by preventing execution of the malware... on-access known-malware scanning can prevent execution as well, at least for known malware (if it's in some obfuscated wrapper like a dropper, then when the malware is 'dropped' it will no longer be obfuscated and that camouflage will no longer be protecting it from the scanner)... application whitelisting can also prevent a great deal of malware (even that which is too new to be considered 'known' yet) from executing (by virtue of preventing everything that isn't already explicitly allowed to execute from executing) so long as the user doesn't give the malware permission to run...

after malware has successfully begun to execute on a system, any certainty about preventing that malware from compromising that system is gone, but that doesn't mean there aren't still possibilities for prevention... it may still be possible to prevent compromise by preventing one or more of the malware's bad behaviours using behavioural blacklisting, behavioural whitelisting, or some combination thereof (all of which falling under the general umbrella of behaviour blocking)... in so far as certain behaviours involve accessing system resources that can have their access restricted, operating as a user who doesn't have access to those resources (ie. running as a non-administrative user, following the principle of least privilege) will prevent a great deal of existing malware from being able to operate properly and thus can prevent compromise...

finally, if malware gets past all those previously described defensive lines, there's still one possible opportunity to full system compromise... if you can avoid the consequences of the malware's behaviour by being lucky enough (or by managing the circumstances well enough) that the malware was actually running in a sandbox rather than on the main host system then it will be the sandbox that gets compromised rather than the main host system... sandboxes are such that their compromise is usually inconsequential because they can be regenerated easily - though extrusion of sensitive data may still be a problem, if the malware was contained withing a sandbox then compromise of the host will have been avoided... otherwise the system will have become compromised, prevention will have completely failed, and it would now be an issue of detecting the preventative failure, diagnosis to determine the extent of that failure, and recovery from it...

you can make any one of these stages your own first line of defense, but only by ignoring earlier opportunities to defend yourself... likewise, you can make any of these your last line of defense, but only by ignoring subsequent opportunities to defend yourself... think about what that statement means: ignoring opportunities to defend yourself... is that really something you want to do? does it sound wise? it certainly doesn't sound that way to me...

teaching bad dogs new tricks

well, april 1st is nearly gone and the internet is still here... conficker's magical doomsday payload didn't materialize - which is good, i would have felt worse if it had...

huh? what? why would i feel bad about such a thing? well it's come to my attention that i may, possibly have contributed to the problem in some small way...

take one autorun worm that employs autoplay social engineering...

add one video that suggests autoplay social engineering was first seen in this worm...

and finally add one blog post published 3 months prior to the discovery of the worm that describes that very feature as a passing remark in the last sentence...

what do you get? a not so great feeling in the pit of my stomach... of course it's probably the height of conceit to imagine that i personally gave this idea to the author(s) of conficker, but autorun worms aren't exactly new in and of themselves so this new behaviour following so close on the heels of my mentioning of it does seem a little bit troubling...

so if i am responsible for that particular feature, my apologies to, well, the entire internet and computer using public... i thought it was an obvious ploy, i assumed it had already been done... it was not my aim to give the bad guys ideas (if anything i'd rather be giving the good guys ideas - if only they'd listen)... i'd like to say that i've come up with safeguards against giving the bad guys ideas in the future, but short of keeping my big yap shut i really can't think of anything...

on the bright side, at least i didn't write and distribute proof of concept attack code that was later used in real malware like some folks i could mention...

what is autoplay social engineering?

autoplay social engineering is a subset of social engineering tricks that specifically utilize the autoplay dialog that windows normally presents when certain types of storage media are inserted into the computer...

related to autorun malware, autoplay social engineering assists in getting the malware executed in situations where autorun doesn't automatically launch the malware as soon as the storage media is inserted into the machine... when the autoplay dialog is displayed it presents the user with a list of options for how the user can view the data on the storage media (such as viewing a slide show if the media contains pictures, viewing a video on the media, opening the drive in explorer, etc) and can also include an option, specified in the autorun.inf file in the root directory on the media, that can literally be anything (including launching of malware processes)... this optional entry would appear at the top of the list and be selected by default when the autoplay dialog opens...

should the option specified in the autorun.inf file present itself as something it's not, such as showing the icon and description identical to that for opening the drive in explorer when it actually runs malware, then that represents a form of social engineering as it's tricking the user into doing something s/he doesn't really want to do...

back to index

failure rates and the law of truly large numbers

probably the best known metric for measuring the effectiveness of an anti-virus product is the detection rate... it's something that's been around for a long time and there are frequent attempts to measure it...

i'm not about to try and suggest something to supplant that metric, just so you know - the detection rate metric has served us well for many years, even if there's a bit of confusion of what actually constitutes a detection rate, and even though there's controversy over how it should be measured and who is qualified to do so...

no, although failure rates are much more straight forward to measure, i'm not about to suggest that they're better in any way - i only want to bring them up in order to provoke thought...

let's say you're an average user and you have anti-virus protecting your computer (possibly at many different levels, like a desktop client, an email gateway scanner run by your email provider, etc)... let's further say that on average your anti-virus product fails to prevent your computer from getting compromised once every 5 years...

now, since it is possible to have anti-virus protecting you at multiple layers and with different optimizations at each layer it's important to define a failure as a piece of malware slipping past all of those layers... maybe that means incident response is required, or maybe you've got some other preventative control that stopped it after (not before) it slipped past your final layer of av prevention (ex. maybe you're a little less than average and are actually running without admin privileges that the malware needed to do it's dirty deed)...

i know what you security practitioners are probably thinking - 'where can i get this magical av product that only fails once every 5 years?'... i'm sure that would make your lives easier, wouldn't it... well right now it's just a hypothetical av product but later on we'll see what we can do...

so now let's say you're a security practitioner, you're part of the IT department of some company... let's further say that at this company you and your immediate coworkers are supporting approximately 2,000 of those same average users and you're using the same anti-virus technology... guess what - you can now expect to see an anti-virus failure once a day! (by the way, when one compromised machine goes out and leads to 5/10/50 more machines on your network getting the same malware, i call that the same event - it's the same failure)...

did the av somehow magically become less effective? no, of course not, it's the same technology - but in this enterprise scenario there are 2,000 times as many opportunities to fail per unit of time than there were in the single user scenario because there are 2,000 times as many users... malware compromise depends in part on decisions made by the user (which should be equally good/bad between the two scenarios) but also on exposure to the malware in the first place which, while it may be regular or even frequent, is an inherently random event... that means even if you could guarantee perfectly predictable (not necessarily correct, just predictable) decision-making from the users (note: you can't actually guarantee this), anti-virus failure events are still at least partially random...

highschool level math tells us that a pair of flipped coins are more likely to have one turn up heads than a single flipped coin would, and the same principle applies to anti-virus failure events - more trials means higher overall probability of the failure event occurring, and more concurrent trials means less expected waiting time between failure events...

now let's think about this in the real world - you security practitioners out there are in a perfect position to know how often your company's anti-virus fails and how many people you're supporting so what's the per-person failure rate of your av?.. at my previous employer we had (to my knowledge) one notable failure in a period of 2 years for a company size of about 20 people (and i helped clean it up)... that's a per-person failure rate of once every 40 years!... now i'm willing to bet that we were an anomaly, a statistical outlier, that the true per-person failure rate is more than once in four decades, but i'm also willing to bet that larger companies with 2,000 people in them do not suffer a new failure every single day - which means their anti-virus has a per-person failure rate that is actually less (and therefore better) than the magical example of once every 5 years...

so take an honest look at how often your anti-virus really fails on a per-person basis... one of the things i've noticed is that a lot of the people who are convinced that av isn't doing a good job anymore are people who are using enterprise experience as their anecdotal evidence - not realizing that the more users you bring into the picture the more the law of truly large numbers works against you... it's not that av actually fails often, it's that failure scales up (and that's true for all failure, not just av failure)...

choosing a good password

recently i've seen two separate videos talking about the problem of choosing a good password - one with michael santarchangelo and one with graham cluley... what bothers me, though, is that they're both repeating relatively old ideas with their pass-phrase method...

here's a different approach to the password choosing dilemma:

don't

you heard me, don't... don't choose a password... the primary reason for people choosing passwords is so that they can have a password that they can remember... this is an obsolete requirement - not because people's memories are that much better (they aren't, if anything the opposite is probably true) but because we now use so many things that need passwords that there's no way we can remember different passwords for all of them and have had to adapt... the poor way most people have adapted is to re-use passwords so that there's less to remember, but the smarter way is to store them so that you don't have to remember them at all...

let the computer generate a password for you using a program like password safe, it will be superior to anything you could choose manually... then store the password in password safe because computer memory is also superior to yours... if it's a password you need when the computer isn't on or a password you need in order to get into the computer in the first place, write it down and stick it in your wallet... if it's a password you need at many computers, you can carry the encrypted password database password safe creates around on a USB flash drive (in fact, password safe itself runs quite nicely from a flash drive without needing to be installed everywhere you need your passwords)...

stop following old password advice that hasn't kept up with the times, adapt to the realities of the today and start using technology (even low-tech technology like a pencil and paper) to make dealing with the password problem easier...

the best laid plans of mice and men often go awry

by now news of the bbc's botnet blunder has spread pretty far... the bbc's actions seem to have violated the law, and there's no question that they were unethical and that prevx's complicity also points to an ethics problem in that company...

many people have written about it already, but alex eckelberry's point about the potential for unintended consequences in taking down a botnet reminded me of a discussion at securosis that i had intended to revisit here but never got around to...

the securosis post in question involved a proposal to set up some system by which an authority could shut down botnets - basically to nullify the legal and ethical hurdles that currently keep most malware researchers from taking down the botnets they're studying...

my two comments on the subject were as follows:

#
kurt wismer Feb 17

you make a compelling argument - botted machines are a public security hazard and some hazards are grievous enough to warrant unauthorized intervention…

i instinctively rebelled against this notion because i don’t like the idea of authorities mucking around on my computer out of some potentially misguided notion that they know better than i do… but i can’t find any flaw in the applicability of your analogy…

the only problem i foresee is that if the bad guys can’t hide their creations behind legal red tape then they’ll hide them behind something equally compelling, like commands to self-destruct and wipe the host machines (to get rid of evidence and also to just be mean) if the network is tampered with… this switch from legal to technical controls may mirror anti-tampering efforts in other domains… if they can figure out a way to make killing the botnet do more harm than good then it will be equivalent to the situation we’re in now and no change in law will affect such a technological adaptation…

#
kurt wismer Feb 17

@rich:
i’m not sure the good it would do would outweigh the bad… when 1,000,000 people suddenly have no operating system, what do you think will happen? steve balmer is already balding, the rest of his hair would be gone the instant microsoft started receiving support calls from all the victims… and that’s just the home users…

what happens when some of those machines are in the enterprise? or in government or military? what if they’re part of critical infrastructure? worse still if it’s in such machines in other countries - taking down the botnet could cause an international incident…

self-destructing botnets are something i wouldn’t want to touch with a 10 foot pole…

currently, ethical malware researchers steer clear of tampering with botnets or the machines they're on - if we change the rules of engagement, if the forces (whatever they may be) that currently prevent us from tampering with botnets went away and we started behaving like the bbc did then the malware authors would have to adapt and self-destructing botnets are an obvious technical approach to regaining the tamper-resistance they currently enjoy thanks to the good behaviour that makes the good guys so 'good'...

ethical hacker not so ethical when hacked

one of the posts to catch my eye during my absence was this post on cd-man's blog about ethicalhacker.net getting compromised...

what struck me about it was that the folks at ethical hacker waited months to inform their membership that they'd been compromised and that users should change their passwords...

now, i suspected some months ago that ethical hacker had either had some kind of breach or had significantly changed their MO when i started getting spam at the address i registered with, but to find out that user's credentials have been in the hands of attackers for 8 months before ethicalhacker.net decided to warn anyone is simply outrageous...

i understand that no site is impenetrable so a breach of this nature is inevitable - i don't have a problem with that... i also understand that most of their users are probably using good password hygiene - however i also know that a surprising number of security folks probably don't... there are many people in the security world who, although they will enforce security policies rigidly within the enterprise, do not take anywhere near the same measures with their own systems at home and so most likely reuse passwords... these people were put in harm's way when the folks at ethical hacker chose sneakiness over transparency...

it's events like this that make me glad i use a different randomly generated password for each site (in addition to the different randomly generated email address i use for each site)... ethical hacker's name is now ironic, much like 'little john' or 'tiny', because they definitely aren't putting their users first...

ok, i'll draw a bunny

yes, this is a fluff post, you have now been warned...

some of you may have noticed i haven't posted in a while... in fact it's been over a month since i last posted anything - my post archive has no entries for february at all... as some of you may recall, anti-malware is not my job - in fact security itself isn't really my job unless you count implementing security features in other software (and even then the only reason i'm the one doing it is because i'm the one who's really familiar with the concepts), so security in general and anti-malware in particular isn't my day job but rather something i've just always had a natural affinity for... that being said, sometimes you need to step back for a while, smell the roses, and gain some perspective, even when it comes to something you're passionate about... ergo, i've been absent...

but today i got an email from cd-man, checking up on me and letting me know that he's tagged me in some internet meme about drawing bunnies... now, i must admit i have a fascination with memes (for purely pragmatic reasons) and how better to study a meme than from the inside by participating in it so i figure why not give it a shot... thus i dug out my highschool sketchbook, an appropriate pencil, found a worthy subject and threw this together (if only i also had a scanner and didn't have to use a crappy digital camera to put it on the web)...



so now i've got to tag 3 people... hmmm, 3 bloggers - jonathan poon, costin raiu, and aaron walters...

does conficker have a silver lining?

don weber posted an intriguing thought about the massive conficker worm actually making the internet more secure...

he's got some sound logic - it does shine the spotlight on the problem and give people who know what to do an opportunity to convince decision makers to do the right thing and that could certainly make people more secure...

trouble is i made the mistake of saying something similar in the previous decade... technically it was more along the lines of 'it would be good if X were bigger/more damaging because then people would sit up and take notice'... as you can imagine then along came something that was bigger/more damaging and people did sit up and take notice... where's the trouble with that you say? i got what i wanted, right?

wrong... a lot of people were negatively affected for what turned out to be a temporary lesson... i'm sorry to say but one of the observations i've made over the past 19 years of following the malware problem is that people largely do not retain the lessons of the past and thus wind up repeating history over and over again...

this case is likely not going to be any different - while don suggests that the efforts put forth as a result of this mass infestation are going to make future mass infestations harder he neglects to mention that there have been plenty of mass infestations in the past whose cumulative effects should have made mass infestation darn near impossible by now if the effect had any kind of staying power...

but the effect doesn't have staying power, it's short lived... there certainly is a window of opportunity for people to push through smart policy/technology changes, but the window is not large - take advantage of it now while it's still open...

test branding fail

another fail, but of a different variety... thanks to pedro bustamante for bringing total protection testing to my attention...

total protection is a snake oil term that sends the wrong message to people and makes them believe they can be totally protected (which obviously they can't)... this is something people in the anti-malware industry should know already...

of course, you could say the same thing about mcafee - who i'm surprised don't own the trademark on this particular instance of snake oil, since they've got a product named total protection...