Monday, January 15, 2007

why 'wipe and reinstall' is wrong-headed

there's been a long running debate about the best way to handle malware... some say using malware removal tools is best while others make a strong argument for wiping the drive and reinstalling from backups or even from original media...

richard bejtlich is a wipe and reinstall from original media proponent*... this is odd because you basically have the guy who popularized the awareness of extrusion in the security context advocating a method that makes determining the extent of extrusion of sensitive data from a malware incident impossible...

you see, historically people have thought of the malware problem as being simply a type of intrusion ('something bad got into my system/network') and that all you needed to do was get rid of the intruder, but now-a-days with the criminally oriented malware (crimeware) out there extrusion of sensitive data ('my passwords/credit info/banking info/etc got out') is increasingly becoming a very real possibility... while prevention is still just concerned about what might get in, recovery must now be concerned both with what got in and what might have gotten out...

back in the day, the one advantage a wipe and reinstall had over the more surgical malware removal was expedience - though never actually necessary, sometimes it was faster/easier to just nuke the drive and rebuild from scratch... no, certainty was not one of it's benefits, certainty was not there - just as richard points out that backups could have been compromised, so too can original media be compromised (it's happened in the past and it will happen again in the future)... as such, certainty in malware recovery is unattainable...

as technology has marched forward, the expedience offered by a wipe and reinstall (or similar methods like restoring drive images) relative to surgical malware removal has only increased but it comes at the cost of masking compromises to assets both on and remote from the affected machine... further, that expedience is very tempting to the lazy and/or ill-informed, it becomes the knee-jerk reaction to even a suspected compromise - after all, why bother with anything else if a wipe and reinstall will make it right regardless? why bother even getting a diagnosis?

the answer to that question, of course, is that without a diagnosis (literally, thorough knowledge) of the malware you can't hope to address the consequences of the malware... you need to know what the malware is, what it can do, how it got in, what it might have leaked out, etc... the wipe and reinstall advice that is generally bandied about trains people not to worry about or even think about those things, it implies that all you need to worry about is getting the intruder out... with thorough knowledge, on the other hand, surgical malware removal (preferably by replacing affected software objects with known clean copies from original media or backups where available, or using a removal tool dedicated to that one malware or it's family, or as a last resort using general purpose removal functionality built into most known-malware scanners) is possible, as is determining the likely entry vector and assets compromised...

admittedly, diagnosis and surgical removal may not be a speedy process... while home user machines are generally not mission critical, businesses/organizations often can't necessarily afford to have a production machine out of commission for the length of time it takes to find out everything you need to know... even then, wipe and reinstall is not the answer - instead create an image of the drive from the compromised system (or remove the physical drive itself and replace it with a fresh one) and then rebuild the system so that it can go back into production while retaining all the information necessary to complete the diagnosis after the fact... this has to be done with the awareness that one is putting the machine back into a potentially compromising situation before figuring out how to prevent subsequent/additional compromise, however...

[*update: apparently, richard bejtlich didn't mean what i thought he meant when he said the safest method of malware removal was reinstallation from original media... apparently richard was talking about the process of actually removing malware abstracted from the broader concept of malware incident response procedures - my apologies for the mix-up, but let the following sink in and you'll understand how the confusion occurred... for most people the process of actually removing malware is their malware incident response procedure - so much so that malware removal has become synonymous with malware incident response (to the point that it's used in preference of malware incident response simply because it's a more familiar term and because it's less jargon-laden)... no one really talks about literal malware removal abstracted from the larger context of malware incident response either because they don't recognize the difference between them or they do but the removal part by itself just isn't that interesting...

so i misinterpreted richard when he actually did talk about the removal part by itself, but i'm fairly certain he in turn misinterpreted the use of malware removal that he was responding to - malware removal certification will almost certainly include more than just getting the bad stuff out...]

6 comments:

Richard Bejtlich said...

I am not advocating reinstallation without investigation. I am saying it is not possible to have a decent sense of certainty that malware has been eradicated without reinstallation.

kurt wismer said...

it is a false certainty, and it still frames the issue in a way that tells people not to worry about anything beyond just getting the intruder out...

whether or not it's what you mean or what you practice, it is how a great many people interpret it...

cdman83 said...

In my opinion we can distinguish two cases: (a) home users and (b) corporate user. Now in the case of home users the "wipe and reinstall" tactic is perfectly warranted IF it is followed by steps like:

-apply all security updates
-make sure Windows Update is turned (assuming it is a windows machine)
-make sure that the Windows firewall is on
-disable the Guest account
-change then name of the Administrator account
-install an AV software
-install the excellent hosts file, even if you are on linux
-try running as limited user

Now in case of corporations you certainly have to go deeper, analyze the penetration point, the type of the malware (was it password stealer? a backdoor?) and so on to estimate the actions you have to take (change all the passwords, etc). This analysis is costly (both in time and money). This is the reason why it's not always needed.

kurt wismer said...

@cdman83:
i disagree - i think the needs you lay out for corporate users are also needs of the home users...

do you think home users don't have to worry about hardening their systems against future attacks? do you think they don't have financial info stolen that they need to know about?

further, the steps you lay out for making wipe and reinstall ok are actually not trivial to follow... you have to apply all the patches to all your applications, you have to remember all the security settings you tweaked, etc... as i tried to say to mike rothman in response to his writeup on the security incite blog, the reimaging he mentions is superior to reinstalling in this regard...

that said, the real problem is how this is all framed... as i've said a couple of times now malware removal has become synonymous with malware incident response, so talking about just getting the bad stuff out abstracted from the larger issue sends the wrong message... that wrong message is trickling down to the trenches where people handing out advice (who you would hope would know better) are telling people who really don't know any better that all they need to do is wipe and reinstall...

cdman83 said...

co.comment actually work! :D

Being a malware analyst I can attest to the fact that doing an in-depth postmortem analysis of a malware infection is very, very difficult. I don't mean to gloat but you have to have a deep understanding of many things starting from assembly, OS internals and networking to be able to put together "the big picture". This is not cheap!

I certainly never said that home users don't have valuable information on their systems, but it is my opinion that very few users would agree to participate in a forensic analysis which could cost them considerable time and money. From a financial point of view it makes much more sense for them to do a wipe and reinstall (or to pay somebody to do a wipe and reinstall) than to pay somebody to make a deep analysis.

kurt wismer said...

@cdman83:
regarding the cost/difficulty in getting 'thorough knowledge' of what happened - there's more than one way to skin a cat....

if the home user is hit with known malware then much of the diagnosis is already done and it's just a matter of finding the details about the malware... if it's unknown malware then chances are the user can get their anti-virus vendor to analyze the malware at no additional cost (av vendors want samples of malware they don't have yet and analyzing what they get is kind of what they do)...