thoughts on metasploit's impact

i listened to the network security podcast #257 this afternoon, specifically because i wanted to hear what martin mckeay, josh corman, and hd moore had to say about metasploit and what josh corman calls HD Moore's Law. there were a lot of mentions of PCI and being 'this tall to ride the internet', but the comment that really caught my ear (i was listening to it rather than reading it after all) was that metasploit allows people to test their security against the attacks that are readily available.

and then a voice in the back of my head said "yeah, but metasploit is what makes those attacks readily available". it's essentially equivalent to saying that the readily available attacks allow people to test their security against the readily available attacks - i believe the way the internet identifies such tautologies these days is by saying "obvious statement is obvious".

one of the interesting things josh corman brought to the conversation was a breakdown of adversary classes (i encourage you to read his post that i linked to above, not only for that breakdown but also a visualization of their relative success rates against a scale of defender strengths) and it occurs to me that, in the absence of metasploit, these so-called readily available attacks that are in the hands of the casual attacker wouldn't generally be in the hands of the casual attacker (and thus wouldn't be the readily available attacks) but rather in the hands of adversaries of a higher calibre.

one thing that isn't really mentioned but seems fairly obvious is that the higher up you go on the scale of adversary classes, the smaller the population will be (the more skills one has the rarer one becomes) and consequently the smaller the aggregate pool of practical targets will be (since there's a limit to what any one given person can pull off in a given period of time, the manpower available to an attacker is a finite resource). that means that in the absence of metasploit, these attacks would be directly impacting fewer systems - probably more important systems, but fewer systems in total.

now before i go any further, lets address an assumption i've made. i think it's an obvious one. you've probably had it on the tip of your tongue for at least the past two paragraphs. the assumption is that in the absence of metasploit nothing else would pop up to take it's place. for my purposes, that's actually not so much an assumption as it is an ideal starting point or degenerate case from which to build a more complex model.

so let's say that another group of well-meaning researchers decided to pick up the gauntlet. i see no intrinsic difference between that hypothetical case and the actual case we have with metasploit right now. that makes it really not that interesting an alternative, because it's not really alternative in any meaningful sense. the more interesting alternative lies in the argument that if the good guys didn't do it, if they were all too principled (for lack of a better word) to follow that path, then the bad guys certainly would.

so in that case let's say a group of ne'er do wells decided to produce a similar tool. would it be the same? would it have the same properties and present the same problems? i would argue that it wouldn't - that the incentives in the underground community are different enough that what would be produced either would not be free (and therefore not available to all casual adversaries) or it would not be as capable as metasploit would have been (perhaps because the best exploits would get held back in order to give the developers a competitive or strategic advantage over the attackers they're helping for free). the motive of doing it for the benefit of everyone (that drives the excellence found in the free edition of metasploit) simply isn't compatible with financially motivated cybercrime. greed and selflessness don't mix, so a criminal-driven equivalent to metasploit wouldn't lower the bar as far as metasploit does.

so what am i trying to say? what am i really getting at with all of this? the TL;DR version is that the metasploit folks are too nice to people, including the bad guys. the notion that metasploit represents the attacks that are readily available suggests to me that they lower the bar too much. no one seems to disagree that metasploit is a tool that is used by script kiddies (among others) and so i'm left to wonder very seriously whether there's an actual legitimate use case for metasploit that involves such a completely unskilled user. leave no user behind? i think under the circumstances an exception deserves to be made.

metasploit open letter revisited

"what we have here is a failure to communicate"

at least, that's what it feels like after fielding a bunch of comments on my last metasploit related post. hd moore specifically felt the need to clarify his view and since it seems that my position could stand some clarification itself i'd like to reiterate my response to him here.

my open letter to the metasploit community was meant to drive home 3 points:

1. AV detection of metasploit output is a problem for anyone trying to use metasploit legitimately and not necessarily appropriate.
2. when people go around saying there's something wrong with AV because it fails to detect metasploit output (in other words, there's something wrong with AV because it fails to do something we've already established would be a problem if it did do) then that actually contributes to the problem by generating market pressure on the AV vendors to correct this supposed problem and add detection.
3. since (1) is a consequence of (2), therefore discouraging (2) should help minimize (1).

(2) is what i saw when i watched john strand's video, but as it happens i've also had a chance to communicate with john and gain a better understanding of where he was coming from (i'd like to think he also gained a better understanding of where i was coming from but that's immaterial for this discussion). i won't divulge the contents of a private communication but i will say that i came away with the impression that he was more interested in showing that there is something wrong with defenses that rely too heavily on what is traditionally considered AV.

if my interpretation is correct then i am in complete agreement with him and i think you'd be hard-pressed to find anyone in the anti-malware community or industry who would disagree. i'd even go so far as to say that his video was an appropriate way of demonstrating that point if that point had been presented in such an unambiguous manner. unfortunately that's not what i took away from it when i first viewed it, and as some of the comments on the 'open letter' post show there are those with a far less nuanced understanding of the topic. i just watched the video again and although i found the phrase that triggered my previous response, in light of my new found understanding i no longer see it the same way.

there isn't anything wrong with AV technology per se (and there are certainly some vendors with more non-signature-related bells and whistles than others), but rather the way people use it (as their one and only defense). it's a poor craftsman who blames his tools, so stop trying to drive screws with a hammer and stop trying to block new/unknown threats with a known-malware scanner.

open letter to the metasploit community

dear metasploit community,

first, please direct your attention to the following video as it demonstrates the very thing i'd like to speak to you about:


as members of the metasploit community, you are no doubt aware of the various legitimate uses for metasploit and the exploits it generates. validating the efficacy of a patch, testing patch deployments, maybe even some penetration testing.

i imagine that you're also aware that some in your ranks have embarked on a wholly misguided ideological quest to highlight the supposed shortcomings of the anti-virus industry using metasploit and it's output.

now let's be clear about something here; we are all in agreement that there are legitimate uses for metasploit's output, therefore that output in general can only be classified as potentially unwanted programs. let's also be clear that the only proper way to address the vulnerabilities that metasploit's output exploits is by patching the vulnerable software. as such, the argument put forward that there's something wrong with anti-virus products that don't detect metasploit output is fallacious on 2 counts: 1) the output isn't necessarily malware (usually only greyware), and 2) anti-virus products are not the proper defense against known exploits (patching is).

there are some concepts in the anti-malware community that some members of the metasploit community may be less aware of than others. one is that there is a countably infinite number of possible programs, and of those a countably infinite number that can do bad things. it is beyond the realm of possibility to detect an infinite number of things so the anti-malware industry has wisely limited it's focus to those that actually exist, to threats that are real, at least as far as malware scanning goes.

another concept which may not be all that apparent is that, while we've faced users of malware creation toolkits for a long time, we have generally lumped them in with the script kiddies due to the complete lack of skill requirements for using such toolkits.

now, even though metasploit isn't a malware creation toolkit (since it's output generally can't be considered malware), it's being used like one in the above video (with the concomitant implication for the user in question). furthermore, this non-malware is being uploaded to virustotal, effectively abusing the service by wasting it's bandwidth on known non-malware, in order to test scanners in a manner that the people who run virustotal themselves say is invalid.

this in turn is putting pressure on av vendors to add detection for metasploit's output, even though that output isn't technically malware, and you know what? as vendors add better and better detection for metasploit's output files, metasploit becomes less and less capable a tool for the legitimate purposes we already identified because anti-virus products will interfere with it's use. how can you use output files from metasploit to validate patch efficacy, test patch deployment, or perform pen-tests if the anti-virus is blocking them? the argument could be made, i suppose, that anti-virus impeding pen-tests is actually a good thing, but it's clear that anti-virus is a fragile defense against exploits when compared to the proper defense of actually patching the vulnerabilities and if AV is interfering with your ability to test if the proper defenses are in place, that really can't be considered a good thing.

with all of this in mind, i would like to ask the metasploit community to do whatever you can to help discourage people from engaging in the behaviour demonstrated in this video. it's arguably worse for the metasploit community than it is for the anti-malware community, but it is definitely bad for both communities and, i think, bad for security in general. no one is helped when people burn one perfectly good tool in order to shame another simply because the tool they're targeting doesn't behave the way they naively think it should.