Wednesday, November 26, 2008

clarification on my morro worse case scenario

well, it looks like a blog conversation may be forming, or perhaps not - we'll see how things go but rich mogull has put up a response to my earlier post on morro, which in turn was partially a response to him (see, a conversation)...

rich doesn't exactly agree with my worse case scenario, but let me be clear it was a worst case scenario (one based in part on the idea rich put forward about microsoft gobbling up the consumer av market) - things can easily go differently if we just keep our eyes open for the signs and avoid them... that being said, the reasons he doesn't agree with me just don't make sense to me...

ignoring whether or not i'm assuming anything about the nature of the av market (granted i don't have the insider knowledge a member of the industry would have, but malware/anti-malware is my main focus as a security blogger), the fact is that there is a non-negligible amount of innovation in it... it may not be a lot (it depends on how you quantify things) but it's certainly not zero... zero innovation is what will happen when there's only one game in town - history has already taught us this and one of the same principals (microsoft) was involved then too...

lets look at some of his specific reasoning:
Morro will be forced to innovate like any AV vendor due to the external pressures of the extensive user base of existing AV solutions, changing threats/attacks, and continued pressure from third party AV.
the problem with this is that rich has already posited the scenario where microsoft gobbles up the consumer av market... what other pressures would it be subject to in that case? there is no extensive user base of existing av 'solutions' (hack, cough, i nearly choked on that term) when microsoft gobbles up the market because there are no other consumer products worth mentioning besides morro... as a result there's no real reason for them to keep on top of the changing threat landscape (anymore than there was for them to keep on top of the changing web landscape) because, once again, they're the only game in town...

Morro will force AV companies to innovate more. Morro essentially kills the signature based portion of the market, forcing the vendors to focus on other areas.
actually, if morro gobbles up the consumer market then whatever other av companies are left will be strictly enterprise av companies and they won't be affected by morro in the least since morro is not an enterprise av product...

there's also the question of ease of evasion... rich is right that it's already pretty easy for anyone to evade the current crop of customer-side scanners... that said, it would still be far easier if there was only one product... it's the difference in complexity of evading a single product versus the complexity of evading all of them - the two scenarios aren't even in the same ball park...

while we're on the topic of low innovation and ease of evasion, however, it seems a good time to mention a rather game-changing innovation that's been popping up in various products recently - scanning in the cloud... panda (not exactly one of the big three) brought this technology to market long before symantec, mcafee, or trend jumped on the bandwagon - but jump they have, and mcafee's artemis has even been included in virustotal... the way i see it this represents a significant innovation and as more and more vendors adopt this approach a number of the currently popular passive evasion techniques (such as targeted attacks and malware q/a) are going to increasingly become obsolete...

so it would seem that the state rich thinks we're currently in (low innovation, easy evasion) is one we may be getting out of, without any help/pressure from a certain known monopolist...

the benefits of scanning in the cloud

now i know a good chunk of the general security industry has been poo-pooing the cloud recently, and normally av is the security industry's favourite whipping boy, so maybe this is just a case of two bad tastes that taste bad together... that being said, the concept has significant promise to take back several tactical advantages that av hasn't had in, well, forever...
  1. signature generation to client update time is reduced/minimized

    usually a good thing and this time it isn't at the expense of q/a because rather than cutting a corner that affects quality they're cutting the corner of updating the client in the first place... instead they'll be updating the cloud which they have direct control over (unlike the client) and then the client doesn't need to detect that it's out of date and try to update itself (and hope updating hasn't foolishly been disabled)... this potentially gives vendors time to do more q/a on their signatures and so reduce the bad signature release rate (though it's already pretty low)...

  2. under-reporting of new samples is reduced/minimized

    under-reporting is the single biggest advantage that targeted attacks have... without the law of large numbers favouring someone noticing there's something fishy about a particular file, that file doesn't get submitted for analysis and nobody gets signature-based detection capabilities for it... in cloud-based scanning, however, just about everything (except those things the user feels are too sensitive to transmit, and as such are probably not malware) should get submitted to the cloud so that is no longer an issue...

  3. greater situational awareness/intelligence than ever before

    data on detections can be correlated and analyzed, etc. providing the potential for virtually every client to become a sensor in a giant honeynet... geographic and demographic trends/patterns in the attacks have the potential to be more easily seen with so much more real-world data, and those are things that can be used to better predict who's at increased risk or maybe even help to pinpoint the source(s) of the attacks...

  4. conventional malware q/a should be entirely thwarted

    the ease with which current malware evades known-malware scanning is on the verge of becoming history... the basic methodology for evasion is to iteratively produce samples and run them past a slew of scanners to see if it's different enough to avoid them all (or enough of them to be valuable)... this worked in the past because malware authors could do this without anti-malware vendors being any the wiser... with a cloud-based scanner you can't fully scan a new malware sample without the vendor getting a copy (either the sample is submitted to the servers controlled by the vendor, or the sample is not submitted and the malware author gets incomplete/inaccurate detectability results) and thus letting the cat out of the bag about not only what your new malware looks like but possibly also what the heck you're up to ("hello, police, i'd like to report a large number of new malicious programs being generated at the following IP address")...

  5. scanner reverse engineering is almost completely nullified

    before what we now know of as malware q/a existed, the more clever malware authors were believed to have reverse engineered various scanners looking for information they could use to make sure their malware would better avoid detection... and even today, those who deal in vulnerabilities (either for the betterment of security or for malicious gain) will analyze scanners looking for flaws that an attacker could take advantage of... with the scanning engine no longer residing on the client computer the only kind of analysis anyone without source code access can do is black-box analysis (and if a botnet can detect attacks and protect itself a cloud-based scanner should conceivably be able to as well)... in this way the scanning algorithm becomes as inscrutable as any server-side polymorphic engine...

Friday, November 21, 2008

badware busters - a 'me too' effort

i read yesterday that the folks at stopbadware.org and consumer reports webwatch are starting a community called badware busters to help ordninary people get malware off their computers...

the stated reasoning is that there's no central place where people can get this help, and they're correct there is no single central place, rather there are dozens of them... some of them even have the slashdot-esque features that badware busters seem to hope will set them apart....

that is if the folks at badware busters are even aware that there are so many communities already doing this... i'm really not sure what they're thinking or how they expect to become the central community for this sort of thing when there's already places like wilders security forums, castle cops, the communities that just about each and ever av vendor seems to set up, the long list of communities you can go to for help with hijackthis logs (and generally don't deal exclusively in hijackthis log analysis), various usenet newsgroups, etc...

dare to dream, guys/gals, but i just can't see you displacing all the other communities that are already out there (and castle cops takes this kind of assistance giving very seriously) and becoming the place to go for all your malware removal needs...

the secret truth about programs

do you know what a program is? are you sure? can you tell the difference between programs and data?

the average person probably thinks of programs as being things installed on their computers that they click on and that subsequently open a window on their computer... somewhat more sophisticated users might be aware of such things as *.exe and *.com files on microsoft platforms, the execute bit on linux, or whatever property tells osx that something is executable on that platform... more technical users like programmers are probably familiar with scripts and may even realize that those are also programs, despite them not resembling anything the average user would consider a program... any computer scientist worth his/her salt, however, knows that none of these are the truth...

if you think you can tell the difference between data and code then you actually don't know what a program is... the truth is that there is no intrinsic difference between data and code (thus, if you think you can tell the difference you're deluding yourself)... all data has the potential to be interpreted as code (and thus be a program), all it needs is the right interpreter to treat it as code (either by design or by accident)...

think of what that means for anything that tries to control what programs do or whether they execute... maybe you can control the actual program, but maybe the best you can hope for is controlling the program's interpreter (be it your web browser, word processor, or some arbitrary system component handling a malformed request)... controlling programs by way of controlling their interpreter is a little like controlling programs by way of controlling the user... if the user or interpreter needs a lot of privileges then the program running in his/her/it's context will have those privileges also...

the classic example of how this is a problem in malware is word macro viruses - sure you can prevent microsoft word from manipulating system files, but you can't reasonably prevent it from modifying other word documents and thereby spreading the malware - ms word is supposed to modify word documents, that's it's job...

Thursday, November 20, 2008

the blacklist value proposition

how do you defend the use of blacklists in the face of seemingly stronger defensive mechanisms like whitelists?

no matter what defensive technology you use there will always be some holes in those defenses... there will always be exceptional cases that your defenses don't currently handle and/or are unsuitable for handling... what's the fastest/easiest way to deal with exceptional cases you want to avoid? yeah, you guessed it, with a blacklist...

let me give you an example: lets say that we have an application whitelist... application whitelists control the execution of some subset of known program types... they're limited to the known types because, well, how do you intercept a kind of execution that's never been seen before?... they're also usually limited to a subset of the known types because developing the technology to intercept and block programs of an arbitrary type (such as script programs for a particular interpreter, or the unanticipated programs that exploit code represents) is not necessarily easy or cheap and for the more obscure types it's often just not worth the investment...

now lets say that a piece of malware is created that exploits this partial coverage of the set of program types... when there's only one such piece of malware, or even just a handful, the benefits of re-engineering the application whitelisting software to be able to cope with this additional type don't justify the costs in terms of time, money, and effort required to do it... when there are so few instances (relative to the billions of programs out there) it is faster, easier, and cheaper to just look for those particular instances (via a blacklist) than to re-engineer the whitelist to handle them... it won't be until the program type in question becomes mainstream that it becomes worth it to add capabilities for it to an application whitelist...

similar scenarios can be constructed for any other type of preventative measure... as such there will always be a need for blacklisting regardless of what other defenses are in play because there will always be a need to deal with emergent exceptional cases as fast and cheaply as possible... even malware blacklists (ie. known malware scanners) themselves have exceptional cases that they can't deal with - that being new/unknown malware... however, as i've stated in the past, novelty is an advantage that wears off, and as far as i can tell it's the only one that does...

Wednesday, November 19, 2008

possible downsides to morro

if you haven't heard the news microsoft is killing onecare and replacing it with a free anti-malware tool probably using the same engine as the current product...

i've written about microsoft's entry into the anti-malware space before and i wasn't very positive about it's chances... microsoft surprised me though, i have to give them credit, and i think it really came down to wooing some of the brighter minds in the av industry away from their then current employers to work on the new microsoft offering (of course ms has also wooed some less scrupulous minds as well)...

that being said there are still some issues to consider... both rich mogull and graham cluley feel this is a positive development for a variety of reasons but rich puts forward the possibility of microsoft bundling the anti-malware software into the OS at some point and basically gobbling up the consumer av market... i doubt you need to be a rocket scientist to see the parallels between that scenario and what microsoft did back in the mid-90's with internet explorer, and i don't think i need to remind anyone that that was actually not good for users (it resulted in microsoft winning the first browser war and then, in the absence of credible competition, they literally stopped development/innovation for years)...

what we don't want or need is for microsoft (or anyone else, technically, though microsoft has the most potential due to their position) to win the consumer anti-malware war in any comparable sense... it's bad on a number of different levels - not only is it likely to hurt innovation by taking out the little guys (who tend to be more innovative and less constrained by the this is the way we've always done things mindset), but it also creates another example of a technological monoculture... granted we're only talking about the consumer market, but the consumer market is the low-hanging fruit as far as bot hosts go and while it may sound good to increase the percentage of those machines running av (as graham cluley suggests) if they're all using the same av it makes it much, much easier for the malware author to create malware that can evade it...

i'm really not sure trading technological heterogeneity (and all the benefits thereof) for a somewhat broader coverage (or even complete coverage) of the consumer market would actually be a good thing, but i am sure i don't want to find out... let microsoft give away their technology if they must, but keep it out of the operating system itself... there are other, safer ways to get anti-malware more broadly deployed...

Tuesday, November 18, 2008

whitelist opinion smackdown

i realize i've been rather quiet as of late - not sure why, perhaps i lost my mojo... anyways, you can all thank cdman for rousing this ogre out of slumber...

in a recent post, cdman lays out his response to randy abrams' post on whitelisting... perhaps it was the hint at the possibility of an ad hominem attack against a fairly well known and long-standing member of the av community (randy was, for a long time, the voice of av from within the belly of the beast - aka microsoft) that piqued my interest, but that wasn't cool so let's move on...

cdman's first substantive beef is the suggestion that whitelisting companies can't do their job without anti-virus software... ignoring the fact that in practice this is actually true (whitelisting companies currently depend on anti-virus software to determine if something is safe to add to their whitelist) lets look at the hypothetical alternatives he suggests - specifically that whitelist vendors could rely on reputation or building the generic malware equivalent of marko helenius' automatic and controled virus code execution system...

relying on reputation offloads the problem of keeping bad software off the whitelist onto the very people providing the bad software... sure people who provide bad software consistently will get a bad reputation and not be trusted, but what about people who only do it once in a blue moon? microsoft releases tons of legitimate and safe software but they have on occasion also distributed virus infected materials... you'd be hard pressed to justify not whitelisting code from microsoft if you were relying on reputation but if you did whitelist all their code you would eventually whitelist something you shouldn't have... furthermore, relying on reputation is precisely the method that customer-generated whitelists are primarily made with, which would make a vendor-generated whitelist using the same technique rather pointless...

next is the idea of building a system to automatically execute samples and perform baseline comparisons to see if the sample compromised the system... and of course this has to be done on a scale sufficient to handle the rate at which sample files are produced (otherwise whitelist vendors wouldn't be able to keep up, much like av vendors supposedly aren't able to)... but have you looked at bit9's (a whitelist vendor) figures? av companies already augment their small armies of malware analysts with automated methods of determining what's bad, and old methods like this are almost certainly among them... if the av vendors can't keep up with the malware then what hope do whitelist vendors have in keeping up with the goodware when it's production rate is (necessarily) several orders of magnitude greater than that of malware? there are all kinds of capabilities peculiar to traditional av companies that whitelist vendors could try to replicate in-house, but the scale of the samples they have to deal with make it impractical for them to do anything other than to replicate the blacklisting capabilities in full in-house and that would mean they would still be using what the general population considers av - it would just be their own...

a third option cdman mentions is using technology like that developed by mandiant... whitelist vendors are unlikely develop such capabilities in-house when it's almost certainly cheaper to buy products/services from others who've already developed those same capabilities, but lets hope in this case they stay away from such ethically questionable companies as mandiant... bad enough that mandiant hires people whose marketability in security is thanks in no small part to their past efforts at making the problem worse, but to then turn around and have some of those same people do essentially the same thing in the company's name at an event like race-to-zero smacks of not just some lapse in HR's judgment but rather of an alignment of moral compasses... perhaps i'm in the minority here, but if a whitelist vendor gets in bed with a company like mandiant i wouldn't touch them with a 10 foot barge pole...

second to the beef about what whitelist vendors would do without av software was cdman's beef with randy's understanding of what actually constitutes a whitelist... i have to admit that my first impression on reading the statement that the TSA implements a whitelist was one of confusion... the most widely known (and reviled) measure the TSA implements is the no-fly list, which is fairly obviously a blacklist... i actually left a comment on randy's original post expressing my confusion but literally as i was writing it it dawned on me that there were other measures implemented by the TSA such as the newly revised rules for flights which basically require one to be granted permission in a 2-stage process before you can fly... of course, as i write this i'm reminded of the various trusted traveler programs that schneier has written about on occasion - those are also whitelists...

despite all the disagreement, though, in the end cdman and randy are actually in agreement about the role of whitelisting - it's simply another layer... both think it's got it's strengths and it's weaknesses, areas where it's more applicable than others, etc.. however, i think randy has once again distilled a complicated topic to a simple analogy when he compares the folks who say whitelists are the end of av with airbags calling seatbelts obsolete... what a clever way to say they're full of hot air...

Sunday, November 02, 2008

suggested reading

  • ThreatBlog » Blog Archive » Giving (Samples) to Charity
    responsible sample handling is very important and, from what i've seen, very misunderstood... i wrote about it myself quite some time ago but it's something that bears repeating and david harley does a good job of explaining what's accepted/expected in the anti-malware industry/community (as opposed to seeming to put one's foot down, as i did)...
  • ICMPECHO · Malware landscape in 2020?
    interesting question/answer about the future of malware from daniel nystrom... there's just one thing i think he missed - if the the past was about fame and the present is about fortune, power/influence seems to be the logical next step... no idea when we'll get there, but as we grow more connected and dependent on technology it will become more and more feasible...
  • hype-free: Popular ideas about AV
    here, cdman reminds me of what i can't stand about slashdot and similar sites - it's a mob of clueless people who somehow manage to influence the thinking of other clueless people... if only there were some way to get them to spread the right idea instead of the wrong one...
  • hype-free: Stepping beyond the vendor-centric security solution
    good post on the importance of understanding the threat and the tools as opposed to listening to marketing (stop listening to marketing!)... the wording reinforces the av = 'blacklist only' impression most people have, but other than that this is a good post with xkcd-style graphics (for people who need diagrams in their explanations - hmmm)...
  • Virus Bulletin : VB2008, Ottawa - conference slides
    no, i'm not going to cherry pick out the best ones... it really doesn't take long to flip through each one... use your best judgment about which are the most interesting to you...
  • Sunbelt Blog: Virus Bulletin 2008 keynote address
    great presentation about the perception of the av industry by both consumers and enterprises... also a great observation on why enterprises are less satisfied - it's scale... everything fails sometimes but when you're dealing with thousands of machines the problem posed by those occasional per-machine failures is magnified... the law of large numbers is not your friend in this context... this is not an easy thing for someone to put into the proper context (unless they've got a really good handle on finite mathematics) so the resulting perceptual bias isn't too surprising...
  • hype-free: Everything is grey
    an unfortunate observation about the virus bulletin conference this year... everything may be shades of gray these days, but i'm still an uncompromising s.o.b. who only sees black and white...