Wednesday, December 31, 2008

the MD5/rogue certificate attack

i'm not going to bother pointing to all the many good stories out there describing the details of how a valid ssl certificate was faked by mounting a 2nd preimage attack on the MD5 hash using a legitimately purchased certificate as the starting point...

i'm just going to point out that, while some people think MD5 was broken in 2004, the fact of the matter is it's use in new systems was deprecated back in 1995, and existing systems should have been moving away from it with all possible haste...

apparently there are ways to make this specific attack impossible without even changing the hash algorithm used (essentially salting the message) and that's certainly a good idea - but still there's no good reason for anything to be using MD5 at this stage of the game... there's been enough time for any legacy system that used it to have been reworked or replaced, and while we should probably start moving away from SHA1 as well (at least to SHA256 until the new SHA3 standard is selected), we should all have moved away from MD5 by now and if you haven't then shame on you...

Saturday, December 20, 2008

returnil give-away at ghacks

i don't normally do posts like this, but in the interests of raising awareness of alternative anti-malware techniques there's a give-away going on at ghacks.net today and they're giving away the premium version of returnil...

returnil is (as far as i know, i haven't tried it yet myself) an instant system recovery sort of sandbox... the free version at least will revert any changes that were made (good or bad) the next time you reboot...

i know that sandboxie was mentioned on the network security podcast and i've mentioned it a time or 2 as well... i think sandboxes are an invaluable security measure and i encourage people to try them out and this returnil give-away is a chance for people to try out the premium version of a sandboxing approach that has a lot of proponents over at wilderssecurity.com...

Thursday, December 18, 2008

the post with no name

well, here i am on the first day of my christmas vacation, trying to do stuff that i didn't have time to do while i was still working...

like blog posts about the distinction between static and dynamic heuristics for martin mckeay (even though that distinction was incidental to the issue i was bringing up)...

since the holiday season is upon us, i thought maybe it might be nice to clarify to martin (and rich mogull before him, and others to whom this will apply equally well) that i wasn't trying to be critical of him personally or attack him... i try not to attack people just for doing/saying something that is technically wrong (ethically/morally wrong, sure, but not technically wrong), everyone makes technical errors... on the other hand, although i don't use people who make technical errors as targets, you folks do make wonderful examples - and examples are an important part of the learning process that (frankly) a lot of people could benefit from...

so, gentle readers who may at some point find themselves the subject of one of my posts, try not to take the fact that i don't usually sugar-coat things too personally - it's just my way...

what are dynamic heuristics?

dynamic heuristics are a branch of heuristic techniques that try to determine if a suspect program is malicious by running the program in a simulated environment and trying to detect the active (dynamic) malicious behaviour(s)...

there are a number of ways to accomplish this, whether it be emulating the program until it reveals the de-obfuscated (and, ideally, previously known) version of itself, or running it in some some sort of sandbox to catalog a broader range of it's behaviours looking for signs of malicious intent...

one of dynamic heuristics' strengths lies in being able to bypass many of the obfuscatory techniques that malware writers use to stymie static heuristic analysis as obfuscation has traditionally needed to be undone at run-time for the malware to operate... unfortunately a number of tricks have been developed by malware writers to try and combat these techniques and they generally exploit a more natural and general weakness involving the fact that for any non-trivial program there are multiple paths that program execution can take but any single invocation of the program will follow only one of those paths... program execution can take different paths (and thus produce different behaviours) depending on any number of different conditions present at run-time so the malicious behaviour a dynamic heuristic engine is looking for may not show up (either by chance or by design) during analysis...

back to index

what are static heuristics?

static heuristics are a branch of heuristic techniques that try to determine if a suspect program is malicious by examining the structure and contents of the program in an inactive (static) state and trying to find (for example) code fragments that have been commonly used for malicious ends in the past...

this type of technique is especially dependent on having detailed knowledge not only of the contents past malware but also of the contents of legitimate programs so as to avoid alerting on the presence of code that, though heavily used by malware, is also common in legitimate software...

one of the strengths of static heuristics is that, unlike dynamic heuristics, it is able to examine multiple possible program execution paths due to the fact that it's looking at all the contents of the program instead of just the code that would get executed during one particular invocation of the program... unfortunately, malware writers have developed a number of techniques to obfuscate their code in such a way as to prevent a heuristic engine from being able to see the actual code and thus preventing it from performing static analysis on that code...

back to index

Monday, December 15, 2008

nuggets of misinformation

over the weekend martin mckeay published a post asking people what free av they used at home... the story is ordinary enough, i'm sure a lot of people out there have faced the problem of what anti-malware software to choose, whether a free one, one of the big name for fee ones, or none at all (and for the record, i'm not in any of those camps)... martin is a well known security blogger and podcaster, he knows about a lot of security and privacy related subjects, but from this fairly informal posting i now know that martin does not know av...

what caught my eye about his post came near the end where martin pointed towards this proactive detection test report as showing how ineffective av really is... for everyone's benefit, tests of proactive protection capabilities are tests specifically designed to bypass the signature-based portion of an anti-malware product so as to test only the heuristic components... that one word - "proactive" - all on it's own would tell someone familiar with this field that the test does not measure the overall effectiveness of products but rather just the effectiveness of a subset of the technologies in those products - and that word was right in the main heading for the report...

reading further (ie. reading the introduction) reveals that the subset of technologies tested is further constrained... the test only measures the effectiveness of static heuristic techniques only, no dynamic heuristics, nothing involving run-time behavioural detection or anything like that... it should be clear that when you're only testing a small part of a product your results won't indicate it's overall effectiveness...[EDIT dec. 16, 2008: turns out i read the intro wrong, however it's still only a test of the heuristic components of anti-virus products rather than of the entire products, and thus not a reflection of their overall effectiveness]

of course if you don't understand the terminology being used and only look at the numbers and the graphs then of course you might think this represents the overall effectiveness - that's probably why martin thinks the effectiveness of av is somewhere between 60% and 80% (not too different from the numbers on the report he points to) when the latest on-demand tests (which still don't include run-time behavioural detection, but do include a broader range of the detective capabilities of the products) performed by both av-comparatives.org and av-test.org place the effectiveness of most products well above 90%...

sadly of all the people who responded to his post, none of them seem to have noticed this interpretation error so far... i'm sure everyone has heard the idiom that there are lies, there are damn lies, and there are statistics... since numbers can be so misleading, it behooves one to familiarize oneself enough with a topic to at least properly interpret those numbers so that you can't be so easily fooled by them...

Sunday, December 14, 2008

does av really suck that badly?

while looking through my rss feeds today i saw this comment rich mogull posted about a week ago (don't ask why it took so long to reach my reader, i don't know)...

primarily it's his observation about malware, anti-malware, and the mac platform and community, but he ends the comment much more generally with this:
To be honest, I think desktop AV sucks in general and isn't nearly as effective as everyone would like us to think.

this is probably a common enough sentiment among the more technically savvy crowd... i wouldn't go so far as to say this is part of the anti-av movement, but rather a consequence of the mismatched expectations people have with regards to anti-virus software and the persistent mischaracterization of av as being solely about virus scanners...

i can understand where the opinion is coming from - if you look just at scanners, and more to the point if you look just at what populist media reports about scanners then the image you get is of something that fails a lot... but here's the main problem with this line of thinking (besides the issue of what av is, which i think i covered adequately before) - no preventative measure is an island complete unto itself...

as i mentioned in my post about the blacklist value proposition, the primary benefit of a scanner is to take care of the exceptions that aren't covered by other measures... scanners have never, ever been the sole preventative measure in play, they've always been complementing something else... even when the only technological measure present was a scanner, there were still procedural measures, there was still common sense, there was still (in the distant past at least) the relative disconnected nature of the computing ecosystem, etc... judging a scanner's effectiveness in isolation as though it were supposed to take care of the entire problem all by itself is like judging how well table salt satisfies your appetite...

the problem is that people think the scanner is supposed to take care of the threat all by itself, and they think that because av marketing departments have been feeding them that line of rubbish for something like two decades now and they aren't really taking many steps to correct the imbalance in the image they're creating and the mismatched expectations they're giving the public... this is why i often frown on marketing, why i've accused those who overuse the concept of protection as snake-oil peddlars, and why i cringe when someone calls a set of security tools a solution...

the problem isn't the technology, the problem is what people understand (or fail to understand) about the technology, and by extension the thing that causes the misunderstanding... as mark linton points out, there is a definite false sense of security being fostered here, and as cd-man suggests in pointing to that same post, that false sense of security is causing harm - possibly even more harm than a scanner can make up for... av companies need to wake up and realize that by allowing their own marketing departments to subtly lie to the public they're going to be shooting themselves in the foot in the long run... by operating in bad faith they are increasingly losing the faith of consumers - and not only will that accelerate when the idea that av sucks makes it into mainstream public consciousness, but it is also very hard to win back once lost...

but back to rich's opinion - i don't think he's entirely wrong, av isn't nearly as good as it's often made out to be, but rich and probably a lot of other people out there are being so profoundly affected by the reality-distortion field put up by av marketing that when they finally start to see a glimmer of reality through a thin spot in the fog bank they see a stark contrast between it and the marketing message and start rejecting everything in the marketing message, even though the best lies are those that are hidden among truths, and come to equally imbalanced conclusions... the opinion is one that smacks of not seeing the whole picture... as i keep saying, what most people call av (the scanner) should be part of a larger whole, not abstracted out on it's own.. further, it's not everyone who wants you to believe av is so good, it's really just marketing (stop listening to marketing; seriously don't even bother rejecting what they're saying, just don't let them affect your thinking at all) and the corporate big-wigs who care more about market share than they do about actually contributing to their customers security and well being (ahem john thompson ahem)... there are plenty of honest, ethical, technical people in the anti-malware industry trying to spread a more balanced message, but they may not be as easy to find as the pitch on the outside of the product's packaging...

Wednesday, December 03, 2008

why perform virustotal-based av tests?

probably most people with any familiarity with the anti-malware field has heard of virustotal.com - for those that haven't, it's an online service that runs the commandline version of a collection of av scanners against submitted samples in order to perform static analysis on them and determine if they're known malware (or perhaps close enough to known malware to be picked up by static heuristics)...

as has been well stated by others - virustotal is for testing samples not for testing anti-malware software... unfortunately that doesn't seem to stop everyone and their grandmother (apparently) from performing comparative and/or effectiveness testing on anti-virus products using the virustotal service...

there are a number of reasons why you shouldn't perform av tests using virustotal, including:
  • those of us who know better will laugh at you - no, seriously, we will
  • virustotal doesn't (can't) include the full detective capabilities of the av products they're using and therefore tests based on their service misrepresent the effectiveness of those products
  • even the people who run virustotal say such testing methodologies are bogus right on their own site
  • retrospective testing already provides results on the effectiveness of av products against new/unknown malware (and it already makes av look pretty bad)


those seem like pretty compelling reasons not to do this kind of testing and yet the practice persists... here are a couple reasons why people might still do it regardless of the reasons not to:
  • it costs too much to do things the right way (proper testing takes a lot of work, time, and resources)
  • people are lazy and virustotal can appear to be a convenient short-cut to getting things done, even though it's really just a short-cut to irrelevance
  • some people seem to be genuinely ignorant of the irrevocable problems with test designs that use virustotal to compare scanners or gauge anti-virus technology
  • related to ignorance but on a grander scale, some people may simply not be capable of designing a scanner test that even flirts with validity, nevermind one that is actually somewhat valid
  • there are some pervasive misconceptions about anti-virus products/technology/vendors/industry that some people have an irrational need to affirm


of course that's just for individual people, when a security company (or worse, an anti-malware company) uses virustotal for quick and dirty av testing then it raises serious questions about the competency of that company's staff... although i have hinted before at the connection between innovation and not being constrained by the 'this is the way we've always done things' mentality, that isn't a license for the security industry to throw scientific rigor out the door...

Tuesday, December 02, 2008

lifehacker's mac anti-virus poll

if there's one thing that never fails to disappoint me it's the failure of the wisdom of crowds principle to work when it comes to malware-related topics, and this ask-the-reader style post on lifehacker lives down to that standard quite well...

you've got some people like astrosmash saying "There are no OS X viruses" - which ignores both the fact that there are in fact os x viruses (osx/leap.a is an overwriting file infecting virus, among other things) and the fact that anti-virus software targets non-viral malware too (of which there has been more than a few for the os x platform)...

you've also got people like texizboy saying:
I don't run A/V on some of my windows machines. All boils down to common sense in my opinion. Webmail services have helped out on this front also, to give credit where it's due, I believe there are less viruses getting around due to them.
despite the fact that email is just one of many different attack vectors that malware have been known to use for some time now, and despite the fact that not all malware is obvious enough for common sense to help (nevermind what they say about common sense)...

then there's people like kilianamphitrite saying:
The real strength of the Mac is that in general, when a Mac is running an untrusted bit of code, it is not doing so with system management privileges. Most of the time (and especially for home systems) Windows users run untrusted code as privileged users.
which incorrectly assumes that you need privileges to do bad things... a lot of windows malware depends on privileged access not because it's necessary for the ultimate goal of the malware, but rather just because such privileged access was almost always there so malware authors didn't have to think of alternatives...

on top of that you've got people like insomniac who says:
The idea of "Mac/Linux/Unix do not have enough market share so people don't develop a virus for them" is only partially true. Unix and Linux based systems are just a lot more difficult to infect because of their architecture and security design than a Windows machine (Vista does a much better job than previous versions of Windows).
which ignores the fact that the first academic treatment of the computer virus phenomenon back in the early-to-mid 80's had viruses successfully spreading in a professionally administered unix environment without aid from privileged users like root...

or how about sverrip who says:
I mainly surf around pages I trust, and don't download and open setup files like "Free-XXX.exe" on my Windows machine.
apparently ignorant of the fact that there is no such thing as safe/trustworthy sites, not to mention ignorant of the existence of the drive-by download vector... even the cbs website can serve malware to unsuspecting victims... and it's not like macs are immune to drive-by downloads - remember the safari carpet bombing flaw?

sad, isn't it? that people believe these fantasies about why they don't need anti-virus software on their mac (or in some cases even pc) computers... i have my doubts as to whether apple's quiet urging of people to use av is going to do anything at this stage of the game... the baseline level of ignorance about malware issues was bad enough but add to that apple's previous arrogance (which no doubt resonated with a lot of their fans) about security and the damage done is all but complete - the only thing left to do is wait for the fallout...

Monday, December 01, 2008

unexpected spam

you may recall me saying here or there that i have a 100% spam free email address... it's an address that i don't give out to people or sites... it's not that the address is unused - i actually use it a lot, but i use it in conjunction with sneakemail.com so it's not my real email address getting spammed - and because i use a different sneakemail address at every site it's no problem to just deactivate or even delete the address and not deal with that site anymore (see my post on avoiding spam)...

so as a result i don't check the spam folder very often - it's almost always empty and when it's not the messages in it are almost always in there erroneously... it's so rare that i actually hand out any address to an organization that will compromise it to spammers (or spam it themselves) that i see more false alarms from the spam filter than i see true alarms...

that all changed with a vengeance today as i found over 50 messages in my spam folder and almost all of them were correctly classified... and wouldn't you know it, the majority were addressed to the sneakemail address that i used for demonstration purposes in this post on phish detection... it certainly took a while for the spammers to find that one (i wonder if they liked the spam poison i laid out as well)...

unfortunately that wasn't the only address that was receiving spam... i don't pretend to know what exactly happened here, but the unique, randomly generated, unguessable address i used to sign up to for ethicalhacker.net has also started receiving spam... the chances of spammers finding that address by enumerating the sneakemail address space are incredibly low (it's a 7 digit base36 number) especially since i have quite a few sneakemail addresses and this is the only one getting spammed by this particular person using the freetellafriend.com service... somehow the folks at ethicalhacker.net let my email address get compromised so you can bet i won't be dealing with them any further (not that i did much there in the first place)...

so anyways, it was quite a shock to see so many spam messages in the spam folder of my spam free email account, but they were all sent to disposable addresses (not the real one) that are no longer reachable so it's all good...

suggested reading

  • As stock market drops malware rises - PandaLabs
    it's not even marginally novel to suggest that malware authors take advantage of the emotional reactions people have to significant world events, be they tsunamis, ice storms, or presidential elections... thus, it it shouldn't come as any great surprise that when people feel their personal finances are vulnerable they are more likely to fall for fake security software, ironically in an attempt to better protect themselves...
  • Schneier on Security: The Neuroscience of Cons
    schneier says fascinating and i have to agree... i just wonder how well this applies to the kinds of social engineering we see in malware and related online threats...
  • ThreatExpert Blog: McColo - Who Was Behind It?
    the story behind the story of mccolo... i wonder what the rap group's connection with the carders was (ie. why were rappers sending out their message for them)...
  • White Listing – The End of Antivirus??? | ThreatBlog
    another balanced whitelisting opinion... i especially like the airbag vs seatbelt metaphor at the end... blacklists and whitelists complement each other, folks - one is not a replacement for the other...
  • Shoulder Surfing a Malicious PDF Author « Didier Stevens
    interesting post about a couple pieces of pdf-embedded malware... the takeaways are 1) malware authors are STILL not great programmers (seems like script kiddies are packaging their 'work' in other files now), 2) incremental update functionality allows script kiddies like this to 'show their work', and 3) script kiddies don't learn from the past (re: formats that contain unique identifiers - might want to ask david l smith about the consequences of that)...
  • Spire Security Viewpoint: WabiSabiLabi Update
    wabisabilabi to close? sounds like good news to me... auctioning off vulnerabilities is a slippery slope that leads to providing a financial incentive for the general public to create attacks, which really isn't a precedent we as a society should be setting...
  • Pirates and Internet Crime - F-Secure Weblog : News from the Lab
    one of the most salient points i've seen made about online crime in a long time... it is indeed as much a social problem as it is a technological one - it is a subset of crime and the reasons for it's existence or it's driving factors are the same as those for conventional crime... so long as those social factors exist so too will crime (both online and offline)...