thoughts on metasploit's impact

i listened to the network security podcast #257 this afternoon, specifically because i wanted to hear what martin mckeay, josh corman, and hd moore had to say about metasploit and what josh corman calls HD Moore's Law. there were a lot of mentions of PCI and being 'this tall to ride the internet', but the comment that really caught my ear (i was listening to it rather than reading it after all) was that metasploit allows people to test their security against the attacks that are readily available.

and then a voice in the back of my head said "yeah, but metasploit is what makes those attacks readily available". it's essentially equivalent to saying that the readily available attacks allow people to test their security against the readily available attacks - i believe the way the internet identifies such tautologies these days is by saying "obvious statement is obvious".

one of the interesting things josh corman brought to the conversation was a breakdown of adversary classes (i encourage you to read his post that i linked to above, not only for that breakdown but also a visualization of their relative success rates against a scale of defender strengths) and it occurs to me that, in the absence of metasploit, these so-called readily available attacks that are in the hands of the casual attacker wouldn't generally be in the hands of the casual attacker (and thus wouldn't be the readily available attacks) but rather in the hands of adversaries of a higher calibre.

one thing that isn't really mentioned but seems fairly obvious is that the higher up you go on the scale of adversary classes, the smaller the population will be (the more skills one has the rarer one becomes) and consequently the smaller the aggregate pool of practical targets will be (since there's a limit to what any one given person can pull off in a given period of time, the manpower available to an attacker is a finite resource). that means that in the absence of metasploit, these attacks would be directly impacting fewer systems - probably more important systems, but fewer systems in total.

now before i go any further, lets address an assumption i've made. i think it's an obvious one. you've probably had it on the tip of your tongue for at least the past two paragraphs. the assumption is that in the absence of metasploit nothing else would pop up to take it's place. for my purposes, that's actually not so much an assumption as it is an ideal starting point or degenerate case from which to build a more complex model.

so let's say that another group of well-meaning researchers decided to pick up the gauntlet. i see no intrinsic difference between that hypothetical case and the actual case we have with metasploit right now. that makes it really not that interesting an alternative, because it's not really alternative in any meaningful sense. the more interesting alternative lies in the argument that if the good guys didn't do it, if they were all too principled (for lack of a better word) to follow that path, then the bad guys certainly would.

so in that case let's say a group of ne'er do wells decided to produce a similar tool. would it be the same? would it have the same properties and present the same problems? i would argue that it wouldn't - that the incentives in the underground community are different enough that what would be produced either would not be free (and therefore not available to all casual adversaries) or it would not be as capable as metasploit would have been (perhaps because the best exploits would get held back in order to give the developers a competitive or strategic advantage over the attackers they're helping for free). the motive of doing it for the benefit of everyone (that drives the excellence found in the free edition of metasploit) simply isn't compatible with financially motivated cybercrime. greed and selflessness don't mix, so a criminal-driven equivalent to metasploit wouldn't lower the bar as far as metasploit does.

so what am i trying to say? what am i really getting at with all of this? the TL;DR version is that the metasploit folks are too nice to people, including the bad guys. the notion that metasploit represents the attacks that are readily available suggests to me that they lower the bar too much. no one seems to disagree that metasploit is a tool that is used by script kiddies (among others) and so i'm left to wonder very seriously whether there's an actual legitimate use case for metasploit that involves such a completely unskilled user. leave no user behind? i think under the circumstances an exception deserves to be made.

my thoughts on benevolent botnets

pete lindstrom recently penned a post on the idea of benevolent botnet... it's not the first time i've seen this topic come, up - martin mckeay posted about a related idea of battling botnets with botnets (presumably one side of that fight would be benevolent botnets)...

my thoughts run something like this - implicit in the idea of the botnet is that the computers that make up the botnet are being remotely controlled without authorization from their owners and therefore no botnet can be considered benevolent...

if the collection of connected computers are being controlled without authorization then you're stealing cycles at the very least, as well as a certain amount of bandwidth in order to communicate with the command and control server...

if the computers are being controlled WITH authorization from their owners then you have a distributed computing project, not unlike seti@home or distributed.net - and you can't really call either of those botnets...

so much like the very act of self-replication makes supposedly good viruses bad, the very act of unauthorized remote control makes supposedly benevolent botnets malicious...

nuggets of misinformation

over the weekend martin mckeay published a post asking people what free av they used at home... the story is ordinary enough, i'm sure a lot of people out there have faced the problem of what anti-malware software to choose, whether a free one, one of the big name for fee ones, or none at all (and for the record, i'm not in any of those camps)... martin is a well known security blogger and podcaster, he knows about a lot of security and privacy related subjects, but from this fairly informal posting i now know that martin does not know av...

what caught my eye about his post came near the end where martin pointed towards this proactive detection test report as showing how ineffective av really is... for everyone's benefit, tests of proactive protection capabilities are tests specifically designed to bypass the signature-based portion of an anti-malware product so as to test only the heuristic components... that one word - "proactive" - all on it's own would tell someone familiar with this field that the test does not measure the overall effectiveness of products but rather just the effectiveness of a subset of the technologies in those products - and that word was right in the main heading for the report...

reading further (ie. reading the introduction) reveals that the subset of technologies tested is further constrained... the test only measures the effectiveness of static heuristic techniques only, no dynamic heuristics, nothing involving run-time behavioural detection or anything like that... it should be clear that when you're only testing a small part of a product your results won't indicate it's overall effectiveness...[EDIT dec. 16, 2008: turns out i read the intro wrong, however it's still only a test of the heuristic components of anti-virus products rather than of the entire products, and thus not a reflection of their overall effectiveness]

of course if you don't understand the terminology being used and only look at the numbers and the graphs then of course you might think this represents the overall effectiveness - that's probably why martin thinks the effectiveness of av is somewhere between 60% and 80% (not too different from the numbers on the report he points to) when the latest on-demand tests (which still don't include run-time behavioural detection, but do include a broader range of the detective capabilities of the products) performed by both av-comparatives.org and av-test.org place the effectiveness of most products well above 90%...

sadly of all the people who responded to his post, none of them seem to have noticed this interpretation error so far... i'm sure everyone has heard the idiom that there are lies, there are damn lies, and there are statistics... since numbers can be so misleading, it behooves one to familiarize oneself enough with a topic to at least properly interpret those numbers so that you can't be so easily fooled by them...