of logic and malware

i've been accused of (among other far more odious things) having formal training/education in the area of logical debating. i don't actually have said training, but i do know a thing or two about logic, so when i read ed moyle's post on security curve about the industry's flawed logic as it relates to malcon it didn't take me long to realize how ed's logic (rather than the industry's) had gone pear-shaped.

to quote:

• Major premise:  All conferences that provide details on how to create malware are a “bad idea”
• Minor premise:  Blackhat/Defcon provide details on how to build malware (e.g. the Invisible Things Blue Pill presented at Defcon 2006; stated goal, “creating 100% undetectable malware”)
• Conclusion:  Blackhat/Defcon is a “bad idea”.

i can use a similar pattern to equally questionable results:

• major premise: cats are furry
• minor premise: marmaduke is furry
• conclusion: therefore marmaduke is a cat

this is the sort of logic statement i recall seeing on tests as a child. often they'd be worded in such a way as to be tricky so as to test our ability to judge the validity of a supposed logical argument. in my example the primary problem is the major premise. while it is true, it's not specific enough to be useful; many things other than cats are also furry.

likewise ed's premises have problems. starting with his minor premise, the details about how to make the blue pill were actually not given out. those details were behind a pay-wall rather than being freely handed out at the talk. furthermore, the classification of the blue pill as malware is questionable at best. just because it's a so-called 'rootkit' doesn't mean it's malware - the reason being that current use of the term 'rootkit' has become so twisted (by which i mean anything that hides things gets called a 'rootkit' now) that even anti-malware products got called rootkits. the blue pill was a novel stealth proof of concept. it could have been used in conjuction with actual malware, but the blue pill itself was not malware.

that tells me, at the very least, that the blue pill was the wrong example for ed to use. we can correct that, however, by using a better example. the race to zero would be a much better example because it involved the creation of actual malware (modifying existing malware to make something that has never been seen before is for all intents and purposes the creation of new malware), which is precisely what malcon aims to facilitate and so makes for a much closer analogy.

unfortunately, even if we replace the reference to the blue pill with a reference to the race2zero, ed's minor premise is still problematic. is the race to zero still not a good enough example? is there a better one? the fact is, no matter what blackhat/defcon presentation you select as an example you will never be able to improve the premise because it would still be just one presentation. blackhat/defcon are about more than just the race to zero or the blue pill. the blackhat/defcon conference pair focus on a wide variety of security issues, many of which not only deserve to be highlighted but also contribute to the betterment of the security condition in three well defined ways. they highlight problems that:

1. should not have happened
2. can be fixed
3. can be avoided in future designs now that we know what to watch out for.

by way of contrast (since ed's argument compares blackhat/defcon to malcon simply by substituting one for the other in his logical framework above), malcon focuses explicitly and exclusively on the advancement of malware creation which is (in general) incapable of providing the same contribution to the security condition. this is the age old distinction between vulnerability research and malware 'research'. with the exception of exploits, malware can't be fixed or avoided because it relies on properties that are intrinsic to the general purpose computing platform.

we also gain no technical benefit by supposedly trying to open a dialog between malware writers and anti-malware researchers.

• for reactive defenses the only prospective benefit would be to help analysts understand the malware. but going back as far as 2006, the average piece of malware could be processed in as little as 5 minutes, so understanding malware doesn't really seem to be something analysts need help with. 
• for proactive defenses the hypothetical benefit would be in letting the analysts know what sort of things are coming so that anti-malware products can catch them before they've even seen them. unfortunately this model is based on predicting the future precisely enough that we'd know specifically what to look for and, as such, is unworkable. the proactive defenses that work are the ones that actually know less, not more, about specific threats whether past present or future (thus why they're called generic techniques).

now, before i stray from the immediate topic any further, let's get back to ed's logical problems. the major premise that "All conferences that provide details on how to create malware are a “bad idea”" is a poor premise as demonstrated by the blackhat/defcon example. one of the necessary properties of a premise is that it's something both parties in an argument can agree on, but this premise is overly broad. as discussed above, blackhat/defcon covers a wide variety of things - can we really say blackhat/defcon is bad as a whole because one of those things might be bad? that seems pretty ridiculous. malcon, on the other hand, is much more narrowly focused on just that one bad thing; so if we rewrite the premise to be more specific, perhaps something like "All conferences that exist solely to provide details on how to create malware are a “bad idea”", then we can include malcon and exclude blackhat/defcon.

now the question one might be asking is, if ed's logic is flawed, what logic would be better? well, for starters i really don't like the major premise, minor premise, conclusion construct - i prefer the premise, inference [, inference...], conclusion construct.

• premise: malware is bad
• inference 1: since malware is bad, creating malware is bad (with the exception of benign exploits)
• inference 2: since creating malware is bad (with one exception), helping others create said malware by doing things that can reasonably be avoided is bad
• conclusion: since malcon will help people create malware by doing something that could reasonably be avoided, malcon is bad.

race to zero in the security special olympics

news has started floating around about a contest in virus obfuscation being held at defcon this year... there have been a couple of mentions of it elsewhere as well, such as robert graham's "race to zero" post, sunnet beskerming's post "defcon competition has antivirus vendors complaining", and even an ars technica post titled "antivirus vendors pan free research from defcon contest" by david cartier...

now i'm obviously opposed to this and think it's irresponsible and unethical... not to mention they've chosen just about the worst malware type to play with - viruses... note to contest organizers, participants, and proponents: in the event that something goes wrong, self-replicators have a tendency to go on and on and on long after they are released or spread (old viruses never die)... they aren't called viruses because they make your computer feel bad, they're called viruses because they spread by themselves just like any infectious biological pathogen... this contest and those in it aim to play with fire...

with that out of the way, it's time i got to debunking a lot of the wrong thought surrounding this contest...

contest organizers (as quoted by pcworld):

Contest organizers say that they're trying to help computer users understand just how much effort is required to skirt antivirus products

if i'm not mistaken one of the key aspects to presenting things is to know your audience... the computer users at defcon already know it's easy to evade a scanner...

race to zero website:

The event involves contestants being given a sample set of viruses and malcode to modify and upload through the contest portal. The portal passes the modified samples through a number of antivirus engines and determines if the sample is a known threat.

this is a logical failure... a modified sample is by definition not a known threat until such time as a signature has been added to the scanner... if you're going to put on this kind of contest you might want to make sure you know what the difference between a known and unknown threat are...

Signature-based antivirus is dead, people need to look to heuristic, statistical and behaviour based techniques to identify emerging threats

this represents a complete failure to understand what the heck they're testing... changing a single bit can often foil signature detection - for contestants to get anywhere in this contest they will have to foil the heuristic, the statistical (also arguably a heuristic), and the behaviour based (heuristic here too) techniques that the scanners also implement... there are no naked known-malware scanners anymore (at least not as far as consumer products go), they all have some kind of heuristic engine in them...

Antivirus is just part of the larger picture,

and this is failure to understand their own messaging/marketing... how can anti-virus be part of the picture if it's dead?... on the one hand they want to tear down the practice of using anti-virus (anti-virus is dead) and on the other they expect people to keep it around as part of the picture... someone needs to make up their mind...

We are not creating new viruses

contrary to their mistaken belief, modifying existing viruses until such time as they're different enough from the original that neither known-malware scanning nor heuristics can recognize them is creating new viruses...

modified samples will not be released into the wild

one wonders how precisely they're going to keep that from happening? they're presenting contestants with samples, not a locked down environment that prevents them from taking their samples with them at the end of the day and doing something stupid/careless/malicious with them after the fact... how do i know this? because providing that kind of environment would be prohibitively expensive and restrictive... it might work in an educational institution (where the costs are offset by student tuition) but not as a contest..

robert graham:

The 'protectors" (product vendors) have big marketing
budgets to tell us their side of the story

it's not their side of the story, it's their attempt to get people to buy their product... on some level we all know that marketing and advertising is just another kind of lying... you don't honestly believe a whopper looks as perfect and juicy in real life as it does on tv, do you? then stop being disingenuous by treating av marketing as anything more factual than that...

We only get one side of the story

that is bullshit... we hear about the failures of anti-virus all the time, we hear about people giving up entirely on anti-virus, and we hear about anti-virus being dead... we get a lot more than just the vendor's side...

Yet, such contests also help customers

defcon isn't a customer education setting, this contest isn't going to teach them anything because they aren't going to be there...

The educating needed here is that the mainstream anti-virus technologies are easily evaded, and that such evasion happens a lot

if it happens so much then why do people need to be educated about it? surely they'll have seen it for themselves or they'll know someone who's seen it and has related the story to them?

Anti-virus vendors publish tests "proving" a 99% detection rate

this is the point where one realizes that robert graham is playing the part of an uninformed crank... anti-virus vendors do not publish tests like that... that kind of self-serving behaviour would absolutely NOT be tolerated by competitors or by the community... vendors point to tests carried out and published by independent 3rd parties...

However, that doesn't apply to customers. Often, the best way to test an anti-virus product is to create your own virus.

this is beyond stupid... you don't test your airbags by smashing your car into things and you don't test your anti-virus product by making new viruses... you leave such testing to the people with the expertise and resources to do it properly... customers generally have neither...

sunnet beskerming:

It should also show up the antivirus tools that
are making use of poor signature detection mechanisms

just like the race to zero website, this is a failure to understand what the contestants will really be bypassing... in order to not be detected by the scanners the samples will have to bypass the heuristic engines in those scanners... getting past a good signature scanner can be as easy as changing a single bit (because a good signature scanner will be very exacting so as to avoid false alarms)...

and those that are using weak heuristics to detect previously unknown malware.

heuristics have to be weakened in order to reduce the number of false alarms to an acceptable level... customers are generally unprepared to resolve potentially false alarms...

It is strange, though, how competitions like CTF, or the recent 0-day competition at CanSecWest, do not attract much complaint, but as soon as antivirus or antimalware tools are targeted it is too much for people

an interesting point, but one that highlights the fact that those other contests revolve around software flaws, whereas showing that new malware doesn't get picked up by blacklists is no more a flaw than notepad's inability to act as a hex editor...

david chartier/ars:

Instead of trying to deride Race to Zero, the AV industry could have a chance at working with the contest to harness what, in reality, could turn out to be some of the best research available on new malicious techniques. "You get what you pay for," as the old saying goes, but in the case of Race to Zero, the AV industry could be passing up a veritable gold mine of free ideas on how to better fight new threats.

except there's nothing for the av industry to learn from this contest... it's already known that malware can be modified, it can be modified in a countably infinite number of ways and if you protect against one the bad guys will just choose (not even find, choose) another... uninformed people think that things would be different if we used heuristics or behaviour blockers or application whitelisting, but the reality is that those can be bypassed in an equally numerous ways... their failures aren't discussed as much because because few people make use of them except for heuristics... and as far as heuristic failures
go people just misinterpret that as a failure in signature scanning, as most people involved in or commenting on this contest have already done...

to repurpose a train of thought from the riskanalys.is blog, the chance of any malware authors coincidentally creating the same malware or using the same modifications as the participants in this contest is basically 0 (n/infinity) so the value of trying to anticipate malware creation/modification techniques and use that knowledge for prevention is also generally 0... conversely, the chance of malware creators/users making use of what is revealed by this contest is greater than zero (because they're lazy, just look at eeye's bootroot to see an example of this having happened) so the value in adding detection for these new samples after the fact (or better still, avoiding the creation of those samples in the first place) is greater than zero...

ultimately i'm reminded once again of something david harley wrote some time ago... he hit the nail on the head when he said the rest of the security industry still doesn't understand av technology, practice, or issues - what people (including the contest organizers) have been saying about this contest proves that much... worse still, robert graham's maligning the credibility of vendors with false statements underscores one of david harley's other points; that the av industry and community remain hugely untrusted...

one has to wonder about the security industry when it doesn't understand one of it's oldest segments (and it's not like the information and people involved aren't available)... the mistrust, on the other hand, is completely understandable under these circumstances - you fear/hate what you do not understand, after all... schneier recently stated that the security industry would be coming to an end and i'll admit i had a bit of a knee-jerk reaction to that (though not so much that i wrote about it - if i did that everytime i disagreed with schneier . . .) but rothman (i think) made a subtle change by saying that security as we know it will come to an end... that's a possibility i almost look forward too - not because i think security should be subsumed by other things, but because the things i'm seeing make me think the security industry has become fundamentally broken (and/or gone mad)... of course, long time readers might recall i have my own prediction about security...