umm, no...
chris hoff makes a valiant attempt at arguing for availability being more important that confidentiality and integrity combined (alluding to a previous discussion on the matter) using the example of the recent conflict between youtube and pakistan that left youtube offline for an hour over the weekend...
unfortunately, what he doesn't mention is that the availability problem that youtube suffered was as a direct result of the corruption of the routing information for youtube... in other words, unavailability was a symptom of an integrity problem...
which reminds me (because anton brought it back to the surface) of another recent article concerning the CIA triad... richard bejtlich posted his observation on the attack trends over the years and from his perspective availability (in the form of bandwidth) was the first to fall victim back in the mid 90's...
i don't think it would be unfair to suggest that richard comes from a network security background and views things through that lens, and i can certainly see where he's coming from given that... once again, though, this looks at a symptom rather than a cause...
causative agents don't generally reside in the network, they reside on devices (though sometimes they are the devices) and when they don't belong on the device they usually qualify as malware... malware goes back quite a bit further than the mid 90's and malware intrusion has always been fundamentally an integrity problem... this is why generic detection was traditionally performed using integrity checking, and even today many current generic controls have an integrity validation component...
of course that's just the lens that i see things through, but i'm not about to turn things around and say that integrity is a bigger deal than availability and confidentiality (especially since i can envision a third perspective where confidentiality is key)... ultimately i think the notion that availability trumps other aspects of security comes from the notion of aligning security with business... the alignment is often one-sided (security changes but management doesn't) and availability (and it's affect on the bottom line) is the thing that management understands best so that's what business-aligned security focuses on most... i wonder what it would be like if aligning security and business was a 2-way street...
devising a framework for thinking about malware and related issues such as viruses, spyware, worms, rootkits, drm, trojans, botnets, keyloggers, droppers, downloaders, rats, adware, spam, stealth, fud, snake oil, and hype...
Wednesday, February 27, 2008
what is integrity checking?
integrity checking is the process of comparing the current state of stored data and/or programs to a previously recorded state in order to detect any changes (and so it sometimes called change detection)...
generally the state information that gets used in the comparison is in the form of a hash of some kind and ideally the baseline state that the current state is compared to was generated when their was nothing wrong with the system...
the reasoning behind this technique is that for anything bad to happen on a system something on that system must change, so then detecting when something bad has happened simply requires detecting changes that shouldn't have happened...
integrity checking is generally considered one of the strongest anti-malware controls, since it has the potential to detect and locate all persistent malware intrusions along with any additional persistent changes to the system that malware could have made... unfortunately it is a detective, rather than preventative control - it can only detect when prevention has failed and so is only good after the fact... it also places a fair bit of burden on the user to be able to tell good changes from bad ones...
back to index
generally the state information that gets used in the comparison is in the form of a hash of some kind and ideally the baseline state that the current state is compared to was generated when their was nothing wrong with the system...
the reasoning behind this technique is that for anything bad to happen on a system something on that system must change, so then detecting when something bad has happened simply requires detecting changes that shouldn't have happened...
integrity checking is generally considered one of the strongest anti-malware controls, since it has the potential to detect and locate all persistent malware intrusions along with any additional persistent changes to the system that malware could have made... unfortunately it is a detective, rather than preventative control - it can only detect when prevention has failed and so is only good after the fact... it also places a fair bit of burden on the user to be able to tell good changes from bad ones...
back to index
Tags:
definition,
integrity checking
Saturday, February 23, 2008
yes mr. rothman, there is a defense against drive-by downloads
in fact, there are several...
mike rothman's days of incite #5 for 2008 was about malware, and while he makes a good point about the inevitability of getting compromised by malware (it's something you really need to be prepared for, though some of us are still waiting) due in part to lack of user input needed for drive-by downloads to work and the difficulty in avoiding exposure to them, he manages to sow some fear, uncertainty, and doubt in the process by stating that there are no defenses against drive-by downloads...
well not to be contrarian, but there are in fact defenses against drive-by downloads... known-malware/exploit scanning in a layered service provider will block the old ones that get re-used (and we know old exploits get re-used) before they reach the browser or any other web content rendering component on the host...
a behavioural HIPS (or possibly even simple application whitelisting, depending on the circumstances) can conceivably stop known or unknown malware injected into the system through a drive-by download...
also, browsing from a sandbox will generally stop most things (known or unknown) coming in through a drive-by download from getting a foothold in the actual host system (leaving the intrusion in the sandbox itself)...
finally, not having the web rendering component that a particular drive-by download exploits installed and/or using a web-content whitelisting technology (such as the noscript firefox extension) can significantly reduce the vulnerable surface you present to the internet and thereby reduce your chances of being compromised...
remember folks, for every measure there is a counter-measure - drive-by downloads are not a magical super-attack that can't be stopped, they're just means of automatically installing malware when you visit a web page and the process can be blocked...
mike rothman's days of incite #5 for 2008 was about malware, and while he makes a good point about the inevitability of getting compromised by malware (it's something you really need to be prepared for, though some of us are still waiting) due in part to lack of user input needed for drive-by downloads to work and the difficulty in avoiding exposure to them, he manages to sow some fear, uncertainty, and doubt in the process by stating that there are no defenses against drive-by downloads...
well not to be contrarian, but there are in fact defenses against drive-by downloads... known-malware/exploit scanning in a layered service provider will block the old ones that get re-used (and we know old exploits get re-used) before they reach the browser or any other web content rendering component on the host...
a behavioural HIPS (or possibly even simple application whitelisting, depending on the circumstances) can conceivably stop known or unknown malware injected into the system through a drive-by download...
also, browsing from a sandbox will generally stop most things (known or unknown) coming in through a drive-by download from getting a foothold in the actual host system (leaving the intrusion in the sandbox itself)...
finally, not having the web rendering component that a particular drive-by download exploits installed and/or using a web-content whitelisting technology (such as the noscript firefox extension) can significantly reduce the vulnerable surface you present to the internet and thereby reduce your chances of being compromised...
remember folks, for every measure there is a counter-measure - drive-by downloads are not a magical super-attack that can't be stopped, they're just means of automatically installing malware when you visit a web page and the process can be blocked...
Tags:
drive-by download,
mike rothman
what is an LSP (layered service provider)?
layered service providers are a technology provided by microsoft to enable developers to insert their programs into the TCP/IP stack in windows... in layman's terms, this means that a developer who implements an LSP is making a program that will sit between the network(internet) and any network clients (such as browsers or email clients) running on the machine...
some malware has been known to be implemented as an LSP in order to intercept outgoing traffic and redirect it or send copies of it to malicious 3rd parties...
spyware isn't the only application for LSPs, however... it's also possible for an anti-malware vendor to implement an LSP that performs known-malware scanning (or sometimes more specifically known-exploit scanning) in order to catch and block attempts to exploit vulnerabilities in browsers and other network clients before they can ever reach those clients...
[see this link for more information on layered service providers]
back to index
some malware has been known to be implemented as an LSP in order to intercept outgoing traffic and redirect it or send copies of it to malicious 3rd parties...
spyware isn't the only application for LSPs, however... it's also possible for an anti-malware vendor to implement an LSP that performs known-malware scanning (or sometimes more specifically known-exploit scanning) in order to catch and block attempts to exploit vulnerabilities in browsers and other network clients before they can ever reach those clients...
[see this link for more information on layered service providers]
back to index
Tags:
definition,
layered service provider,
lsp
what is behaviour blocking?
behaviour blocking is the process of monitoring the behaviour of software running on a system and stopping any behaviour that is considered unauthorized or bad...
in the distant past behaviour blockers blocked a relatively simple list of behaviours without regard for which program was performing them... in effect, they enforced simple behavioural blacklists... even with a simple list of behaviours there was no behaviour that was unique to malware so there were times when the behaviours being intercepted needed to be allowed to continue - leading behaviour blockers to continually ask the user whether this or that behaviour should be allowed...
over the years the concept has become much more sophisticated and configurable - monitoring more types of behaviours and allowing certain behaviours to go unchallenged when performed by programs that are authorized to perform them (in essence adding behavioural whitelisting to it's repertoire) while blocking others automatically... this type of sophisticated behaviour blocker is now more widely referred to as an HIPS (though it's not the only thing referred to as such)...
back to index
in the distant past behaviour blockers blocked a relatively simple list of behaviours without regard for which program was performing them... in effect, they enforced simple behavioural blacklists... even with a simple list of behaviours there was no behaviour that was unique to malware so there were times when the behaviours being intercepted needed to be allowed to continue - leading behaviour blockers to continually ask the user whether this or that behaviour should be allowed...
over the years the concept has become much more sophisticated and configurable - monitoring more types of behaviours and allowing certain behaviours to go unchallenged when performed by programs that are authorized to perform them (in essence adding behavioural whitelisting to it's repertoire) while blocking others automatically... this type of sophisticated behaviour blocker is now more widely referred to as an HIPS (though it's not the only thing referred to as such)...
back to index
Tags:
behaviour blocking,
definition,
HIPS
what is application whitelisting?
application whitelisting is, as the name suggests, an example of whitelisting whereby only applications that are authorized get to be executed/interpreted/run...
in the case of application whitelisting the whitelist itself is the list of authorized programs (generally with some integrity information like a cryptographic hash to ensure that a given program is actually the same as the one on the list rather than just having the same name)...
if the whitelist itself is provided by a vendor then it will necessarily be very large (as they have to account for all good programs that anyone might use) and may not include everything a user needs... if, on the other hand, the whitelist is provided by the user him/herself then it will likely only contain the programs that are relevant to the user, but the user will have to decide for him/herself which programs are trustworthy enough to put on the list... either way, the whitelist would need to be updated at least as often as the user's set of authorized software changes...
the anti-malware applications of this technique rely on malware never getting on the whitelist... so long as the whitelist remains malware-free then (barring exotic execution) malware shouldn't be able to run... a second, lesser understood requirement is that the application whitelist be able to recognize everything that qualifies as a program, lest some program types get executed without whitelist-based oversight...
back to index
in the case of application whitelisting the whitelist itself is the list of authorized programs (generally with some integrity information like a cryptographic hash to ensure that a given program is actually the same as the one on the list rather than just having the same name)...
if the whitelist itself is provided by a vendor then it will necessarily be very large (as they have to account for all good programs that anyone might use) and may not include everything a user needs... if, on the other hand, the whitelist is provided by the user him/herself then it will likely only contain the programs that are relevant to the user, but the user will have to decide for him/herself which programs are trustworthy enough to put on the list... either way, the whitelist would need to be updated at least as often as the user's set of authorized software changes...
the anti-malware applications of this technique rely on malware never getting on the whitelist... so long as the whitelist remains malware-free then (barring exotic execution) malware shouldn't be able to run... a second, lesser understood requirement is that the application whitelist be able to recognize everything that qualifies as a program, lest some program types get executed without whitelist-based oversight...
back to index
Tags:
application whitelist,
definition
what is known-malware scanning?
known-malware scanning is the process of scanning a computer resource (be it a file, an email, an entire disk, etc) for malware that matches a malware signature and is therefore known to the scanner (hence 'known-malware' scanning)...
known-malware scanning (more often referred to as virus scanning despite the fact that scanners have been detecting non-viral malware for a long time now) is a blacklist-based technique for detecting/blocking known bad programs... the blacklist itself is the set of signatures the scanner uses to try and match samples against...
known-malware scanning is obviously only effective against malware that is known...
back to index
known-malware scanning (more often referred to as virus scanning despite the fact that scanners have been detecting non-viral malware for a long time now) is a blacklist-based technique for detecting/blocking known bad programs... the blacklist itself is the set of signatures the scanner uses to try and match samples against...
known-malware scanning is obviously only effective against malware that is known...
back to index
Tags:
definition,
known-malware scanning
what is a sandbox?
in the anti-malware sense a sandbox is a (generally) simulated computer environment in which untrusted software (ie. potential malware) can be run while still keeping the trusted host environment isolated from it...
a sandbox can be implemented in more than one way, the most well known is probably the virtual machine... a virtual machine based sandbox is one where the computer hardware itself is simulated and an OS and other software run on top of that simulated hardware...
another type of sandbox is one that simulates access to resources (such as files or the registry) of an existing system by trapping attempts by programs running in the sandbox to access those resources and redirecting those access attempts to copies residing within the sandbox environment... this uses less resources than a traditional virtual machine and can be used in a more transparent way by end users because it doesn't set up a completely separate system which may or may not be similar to the host system they're likely familiar with (nor do they have to try to wrap their heads around the idea of a computer within a computer)...
a third (but much more primitive) technique which could arguably be considered a kind of sandbox is the use of different drive images to separate the main computing environment from other environments used for running untrusted code... this has the problem that the environment for trusted programs and the one for untrusted programs can't be in operation on the same computer at the same time, which makes it a little inconvenient if you only have one physical system...
ultimately these technologies are all meant to try to replace the original practice of using a physically separate computer dedicated to testing... a second physical computer for untrusted programs is probably the least convenient way to isolate a trusted environment from untrusted code but it is also probably the most secure...
back to index
a sandbox can be implemented in more than one way, the most well known is probably the virtual machine... a virtual machine based sandbox is one where the computer hardware itself is simulated and an OS and other software run on top of that simulated hardware...
another type of sandbox is one that simulates access to resources (such as files or the registry) of an existing system by trapping attempts by programs running in the sandbox to access those resources and redirecting those access attempts to copies residing within the sandbox environment... this uses less resources than a traditional virtual machine and can be used in a more transparent way by end users because it doesn't set up a completely separate system which may or may not be similar to the host system they're likely familiar with (nor do they have to try to wrap their heads around the idea of a computer within a computer)...
a third (but much more primitive) technique which could arguably be considered a kind of sandbox is the use of different drive images to separate the main computing environment from other environments used for running untrusted code... this has the problem that the environment for trusted programs and the one for untrusted programs can't be in operation on the same computer at the same time, which makes it a little inconvenient if you only have one physical system...
ultimately these technologies are all meant to try to replace the original practice of using a physically separate computer dedicated to testing... a second physical computer for untrusted programs is probably the least convenient way to isolate a trusted environment from untrusted code but it is also probably the most secure...
back to index
Tags:
definition,
sandbox
what is a whitelist?
in contrast to the blacklist, a whitelist is a list of known good things that we don't need to avoid, block, or weed out... a whitelist is generally used in situations where nothing is allowed except those things on the whitelist... this is sometimes referred to as 'default deny'...
a classical real world example of a whitelist is the guest list for an exclusive social function (where you don't get in unless you're on the list)... further examples include the list of things you're allowed to put in your recycling bin, the list of foods and drugs approved by the FDA, or the set of people you'd trust with a secret...
in the anti-malware context the most well known example (to average users) of a whitelist is the list of applications allowed by a software firewall to connect to the internet... you can also have a list of applications allowed to execute (or perform any other function), a list of websites allowed to execute scripts in your browser, a list of email senders allowed past your spam filter, etc...
back to index
a classical real world example of a whitelist is the guest list for an exclusive social function (where you don't get in unless you're on the list)... further examples include the list of things you're allowed to put in your recycling bin, the list of foods and drugs approved by the FDA, or the set of people you'd trust with a secret...
in the anti-malware context the most well known example (to average users) of a whitelist is the list of applications allowed by a software firewall to connect to the internet... you can also have a list of applications allowed to execute (or perform any other function), a list of websites allowed to execute scripts in your browser, a list of email senders allowed past your spam filter, etc...
back to index
Tags:
definition,
whitelist
what is a blacklist?
a blacklist is, simply put, a list of known bad things that we would like to avoid, block, weed out, etc... a blacklist is generally used in situations where everything is allowed by default except those things on the list... this is sometimes referred to as 'default allow'...
a classical real world example would be the TSA's no-fly list, an example that's often dredged up to support the argument that blacklists are themselves bad... another example would be the registered sex offenders list, a third would be the list of things you shouldn't take when you're pregnant, and a fourth would be the list of things you tell your child not to do... the argument against blacklists in general starts to fall down when you start to consider those kinds of examples - whether a particular blacklist is bad or not is not determined by the mere fact that it's a blacklist...
in the anti-malware context, the most well known example of a blacklist is a known-malware scanner (more commonly referred to as a virus scanner)... that's just a blacklist of bad programs, however... you can also have lists of bad behaviours, behaviour combinations, bad websites, bad (spammy) words in emails, etc...
back to index
a classical real world example would be the TSA's no-fly list, an example that's often dredged up to support the argument that blacklists are themselves bad... another example would be the registered sex offenders list, a third would be the list of things you shouldn't take when you're pregnant, and a fourth would be the list of things you tell your child not to do... the argument against blacklists in general starts to fall down when you start to consider those kinds of examples - whether a particular blacklist is bad or not is not determined by the mere fact that it's a blacklist...
in the anti-malware context, the most well known example of a blacklist is a known-malware scanner (more commonly referred to as a virus scanner)... that's just a blacklist of bad programs, however... you can also have lists of bad behaviours, behaviour combinations, bad websites, bad (spammy) words in emails, etc...
back to index
Tags:
blacklist,
definition
what is anti-malware / anti-virus software?
anti-malware is basically the idea of being against malware, it's creation, it's malicious dissemination, etc... anti-malware software, therefore, is software meant to stop, detect, or otherwise clean up after malware... it's an umbrella term that covers basically all technologies that can be used in the fight against malware...
the term anti-virus arose at a time when almost all malware was viral in nature (in fact at a time before there was even a need for an umbrella term like malware) and so anti-virus software was anti-malware software for the single type of malware that was an issue... over time malware has diversified into many different types and as that has occurred anti-virus software has branched out into dealing with those other types as well... further, the underlying concepts behind virtually all of today's anti-malware technology originally came from the anti-virus field... as such anti-virus is basically a synonym for anti-malware...
back to index
the term anti-virus arose at a time when almost all malware was viral in nature (in fact at a time before there was even a need for an umbrella term like malware) and so anti-virus software was anti-malware software for the single type of malware that was an issue... over time malware has diversified into many different types and as that has occurred anti-virus software has branched out into dealing with those other types as well... further, the underlying concepts behind virtually all of today's anti-malware technology originally came from the anti-virus field... as such anti-virus is basically a synonym for anti-malware...
back to index
Tags:
anti-malware,
anti-virus,
definition
Monday, February 18, 2008
when all you have is a hammer, everything looks like a nail
i've got a bone to pick with the av industry (don't look so surprised, i've been critical of them before) that has been stewing for a while now and recent conversations i've had elsewhere have convinced me that it's time to say something...
the problem is the apparent near-exclusive focus on known-malware scanning... don't get me wrong, old malware certainly lingers and known-malware scanning is a quick and easy way to make old malware a non-issue in your environment, but that only works for old malware...
known-malware scanning is not now, nor has it ever been the appropriate tool for dealing with new/unknown malware... for the armchair strategists out there, imagine a situation where you have no intelligence on enemy and then imagine trying to use tactics that require such intelligence in order to be effective... the very notion of using known-malware techniques against unknown malware is an oxymoron - and yet as the set of malware that qualifies as new/unknown at any given time increases, the most obvious reaction from vendors has been to release signatures faster... it's as if they believe they can somehow change the character of the problem of new malware so that known-malware scanning can be the 'solution' to this part of the malware problem too... like they can fix the problem that prevents known-malware scanning from working on unknown malware...
but of course, they don't actually accomplish that... they narrow the window of opportunity a bit for some of the malware, but not enough and certainly not entirely... no matter how quickly signatures are released there's going to be a period of time after the malware's release where it qualifies as unknown and no known-malware technique is going to be suitable for it... furthermore, as the rate of malware production increases the total number of malware samples in that unknown-malware window will increase... while it's certainly necessary to release more signatures per week/month/whatever, getting them out the door faster instead of focusing on more appropriate ways of dealing with the unknown malware problem isn't helping anyone...
instead what it accomplishes is: to cut quality assurance corners (which is the cheapest and easiest way to speed up delivery of anything) resulting in bad signatures that hose customer systems, to make the industry look like a one-trick pony by virtue of the ever escalating effort applied to this single technology, to reinforce in dumb customers the notion that they are staying on top of the problem just by keeping their scanner up to date, and to make smart customers question whether anti-virus has any value at all...
oh, and if you think heuristics are a known-malware scanner's saving grace then you haven't been reading the same retrospective scanner test results i have... the latest retrospective report available from av-comparatives.org (november '07 at the time of writing) shows that by and large the ability of scanners to detect malware using heuristics alone is dismal... only 3 of the 13 products tested managed to score above 50% and even then the best was only 81% (and that product was penalized in the overall rating due to a high false positive rate)... sure 81% is pretty good for heuristics, but it's not good enough to rely on exclusively for protection from new/unknown malware and most were less than half that effective...
av vendors need to start making a much more visible effort in the realm of generic detection techniques, they need to empower people to use those techniques, and they need to stop promulgating this farcical 'solution' snake-oil that makes people think one thing is all you need and instead represent their technologies as tools... that way it's more intuitive that different technologies do different things and that it's often desirable to utilize more than one because there's more than one job to do (more than one problem to deal with)... it makes no intuitive sense to need more than one solution (and believe me when i say people will view the different parts of a security suite as more than one solution) but it makes perfect sense to have a toolbox with more than one tool in it...
the problem is the apparent near-exclusive focus on known-malware scanning... don't get me wrong, old malware certainly lingers and known-malware scanning is a quick and easy way to make old malware a non-issue in your environment, but that only works for old malware...
known-malware scanning is not now, nor has it ever been the appropriate tool for dealing with new/unknown malware... for the armchair strategists out there, imagine a situation where you have no intelligence on enemy and then imagine trying to use tactics that require such intelligence in order to be effective... the very notion of using known-malware techniques against unknown malware is an oxymoron - and yet as the set of malware that qualifies as new/unknown at any given time increases, the most obvious reaction from vendors has been to release signatures faster... it's as if they believe they can somehow change the character of the problem of new malware so that known-malware scanning can be the 'solution' to this part of the malware problem too... like they can fix the problem that prevents known-malware scanning from working on unknown malware...
but of course, they don't actually accomplish that... they narrow the window of opportunity a bit for some of the malware, but not enough and certainly not entirely... no matter how quickly signatures are released there's going to be a period of time after the malware's release where it qualifies as unknown and no known-malware technique is going to be suitable for it... furthermore, as the rate of malware production increases the total number of malware samples in that unknown-malware window will increase... while it's certainly necessary to release more signatures per week/month/whatever, getting them out the door faster instead of focusing on more appropriate ways of dealing with the unknown malware problem isn't helping anyone...
instead what it accomplishes is: to cut quality assurance corners (which is the cheapest and easiest way to speed up delivery of anything) resulting in bad signatures that hose customer systems, to make the industry look like a one-trick pony by virtue of the ever escalating effort applied to this single technology, to reinforce in dumb customers the notion that they are staying on top of the problem just by keeping their scanner up to date, and to make smart customers question whether anti-virus has any value at all...
oh, and if you think heuristics are a known-malware scanner's saving grace then you haven't been reading the same retrospective scanner test results i have... the latest retrospective report available from av-comparatives.org (november '07 at the time of writing) shows that by and large the ability of scanners to detect malware using heuristics alone is dismal... only 3 of the 13 products tested managed to score above 50% and even then the best was only 81% (and that product was penalized in the overall rating due to a high false positive rate)... sure 81% is pretty good for heuristics, but it's not good enough to rely on exclusively for protection from new/unknown malware and most were less than half that effective...
av vendors need to start making a much more visible effort in the realm of generic detection techniques, they need to empower people to use those techniques, and they need to stop promulgating this farcical 'solution' snake-oil that makes people think one thing is all you need and instead represent their technologies as tools... that way it's more intuitive that different technologies do different things and that it's often desirable to utilize more than one because there's more than one job to do (more than one problem to deal with)... it makes no intuitive sense to need more than one solution (and believe me when i say people will view the different parts of a security suite as more than one solution) but it makes perfect sense to have a toolbox with more than one tool in it...
Tags:
anti-malware,
anti-virus,
strategy
why proactive anti-malware controls are dumb
no, i'm not trying to insult proponents of proactive techniques... i'm not trying to say that using proactive controls is dumb, but rather that the controls themselves are dumb...
the diametrical relationship between proactive and reactive preventative controls can be considered equivalent to the one between generic and malware-specific (aka known-malware) techniques respectively... to be truly proactive means to be able to do something about or block a piece of malware before anyone knows anything about it - therefore it can't be malware-specific because a malware-specific control can't be built until after knowledge of the malware is acquired...
therefore proactive/generic controls lack the knowledge component that reactive/malware-specific techniques have and so are stupid... by extension, built-in knowledge is a luxury only reactive/malware-specific techniques can have... of course it is entirely possible to have a little bit of both and provide some shades of gray between these black and white polar opposites...
being stupid isn't necessarily a bad thing in a control, mind you... sharks are often referred to as being perfect killing machines but you won't catch anyone accusing them of being unusually intelligent... so using a dumb control can still be smart if the control is effective... in fact, one could even say that using a dumb control effectively requires intelligence because it doesn't come with any of it's own...
the diametrical relationship between proactive and reactive preventative controls can be considered equivalent to the one between generic and malware-specific (aka known-malware) techniques respectively... to be truly proactive means to be able to do something about or block a piece of malware before anyone knows anything about it - therefore it can't be malware-specific because a malware-specific control can't be built until after knowledge of the malware is acquired...
therefore proactive/generic controls lack the knowledge component that reactive/malware-specific techniques have and so are stupid... by extension, built-in knowledge is a luxury only reactive/malware-specific techniques can have... of course it is entirely possible to have a little bit of both and provide some shades of gray between these black and white polar opposites...
being stupid isn't necessarily a bad thing in a control, mind you... sharks are often referred to as being perfect killing machines but you won't catch anyone accusing them of being unusually intelligent... so using a dumb control can still be smart if the control is effective... in fact, one could even say that using a dumb control effectively requires intelligence because it doesn't come with any of it's own...
Saturday, February 09, 2008
the state of anti-virus
there have been some posts and discussions recently about anti-virus that i thought were interesting (and not just because my name was invoked) and rather that split my thoughts between multiple sites i've opted to combine everything here... thankfully most involved welcome open debate on the subject, though others not so much - but refusing to believe malware and vulnerabilities are orthogonal to one another kinda limits where the discussion can go, so oh well...
so i'm going to start with michael's post over at mcwresearch.com... now i hope michael doesn't take this the wrong way, but i quite enjoyed the irony of his post's title - "AV must innovate or die"... michael of course comes from a behaviour-based HIPS background and behaviour-based HIPS (or at the very least it's precursor) began it's life in an anti-virus product...
furthermore, as i've mentioned before, anti-virus isn't just blacklist-based scanners... rather it's an archaic term for anti-malware and since HIPS is itself a type of anti-malware technology, HIPS is fundamentally part of the same industry that michael is criticizing - which means he himself is making that innovation happen and making sure 'anti-virus' doesn't die...
of course, when michael used the term anti-virus, he really was just talking about known-malware scanners and criticizing them for not being able to do what is totally outside the scope of a known-malware scanner - block unauthorized behaviour... that's actually a pretty unfair criticism since he's only considering the scanner technology itself and no technology (not scanners or HIPS) is perfect, they all do different things and they all have their limitations... he's mentioned in the past that anti-virus needs to get it's peanut butter mixed in with IPS's chocolate so i'm going to give michael a treat... symantec has HIPS, mcafee has HIPS, trend micro has HIPS, sophos has HIPS, kaspersky has HIPS built into KIS, f-secure has HIPS, panda has HIPS, etc... perhaps in the future michael will consider all the technologies that traditional av vendors make available so that he doesn't have to compare apples to oranges anymore...
the other post that caught my eye was this one by lonervamp (he's also a michael but let's not get ambiguous) about where he sees the state of anti-virus being... he also refers to anti-virus in a strictly known-malware scanning context and discusses the role it's supposed to play in a defensive strategy and how it's benefits have waned with the development of procedural and technological layers that complement it's use... i would actually tend to agree with that observation, but i'd also note that being complementary is a commutative property - known-malware scanning complements those other layers as well... could you go without scanning entirely? sure, just like you could use only scanning, but they're better together than they are on their own... that the benefits of known-malware scanning wane in the presence of complementary layers is entirely understandable and expected as the layers in a multi-layered defense will generally partially overlap each other... it's because of that overlap that the benefits of any individual layer may wane, but a layer isn't redundant or useless unless it's completely overlapped by the other layers...
so a question that is perhaps of interest to both posters is at what point does known-malware scanning become entirely redundant?... personally, i don't think it ever does... any given technology has an achilles heel that can allow a piece of malware to get past it... for known-malware scanning this achilles heel is novelty - new/unknown malware is clearly not something known-malware scanning can deal with... other technologies have their own weaknesses too but as weaknesses go novelty is unique because it's the only one that wears off... the media makes quite a big deal out of all the new malware being created and how it keeps getting past known-malware scanners, and at the same time it rarely mentions cases of malware incidents involving old malware... this leads to the erroneous impression that one needn't worry about old malware... old malware still pops up for a variety of different reasons and although it may use tricks that can still allow it to get past other technologies, by the time it gets old it's not going to be getting past a known-malware scanner anymore...
lonervamp also made a distinction between home use and business use, suggesting that known-malware scanning was more likely to be displaced in the business than in the home... i can certainly see that - one of the strengths of known-malware scanning is that the expertise that makes it work is embedded in the product so that the user is relieved of having that burden... many other technologies require knowledge and skill from the customer that may or may not be there... in a business you're much more likely to have trained, knowledgeable individuals (the IT staff) who can make the intelligent decisions necessary for products with more client side configuration to work effectively... that means that alternative technologies become a more feasible option in that environment...
that isn't to say i'd suggest dropping known-malware scanning though... even in a business i think it has it's place... i don't think it should be the only type of technology used, mind you - there are many types of technology available (sometimes even from a single vendor) and they're just waiting to be used... if i were going to suggest anything it would be more comprehensive layering, not simply different layers... the goal should be better overall coverage and substituting one layer for another isn't necessarily the way to get there...
so i'm going to start with michael's post over at mcwresearch.com... now i hope michael doesn't take this the wrong way, but i quite enjoyed the irony of his post's title - "AV must innovate or die"... michael of course comes from a behaviour-based HIPS background and behaviour-based HIPS (or at the very least it's precursor) began it's life in an anti-virus product...
furthermore, as i've mentioned before, anti-virus isn't just blacklist-based scanners... rather it's an archaic term for anti-malware and since HIPS is itself a type of anti-malware technology, HIPS is fundamentally part of the same industry that michael is criticizing - which means he himself is making that innovation happen and making sure 'anti-virus' doesn't die...
of course, when michael used the term anti-virus, he really was just talking about known-malware scanners and criticizing them for not being able to do what is totally outside the scope of a known-malware scanner - block unauthorized behaviour... that's actually a pretty unfair criticism since he's only considering the scanner technology itself and no technology (not scanners or HIPS) is perfect, they all do different things and they all have their limitations... he's mentioned in the past that anti-virus needs to get it's peanut butter mixed in with IPS's chocolate so i'm going to give michael a treat... symantec has HIPS, mcafee has HIPS, trend micro has HIPS, sophos has HIPS, kaspersky has HIPS built into KIS, f-secure has HIPS, panda has HIPS, etc... perhaps in the future michael will consider all the technologies that traditional av vendors make available so that he doesn't have to compare apples to oranges anymore...
the other post that caught my eye was this one by lonervamp (he's also a michael but let's not get ambiguous) about where he sees the state of anti-virus being... he also refers to anti-virus in a strictly known-malware scanning context and discusses the role it's supposed to play in a defensive strategy and how it's benefits have waned with the development of procedural and technological layers that complement it's use... i would actually tend to agree with that observation, but i'd also note that being complementary is a commutative property - known-malware scanning complements those other layers as well... could you go without scanning entirely? sure, just like you could use only scanning, but they're better together than they are on their own... that the benefits of known-malware scanning wane in the presence of complementary layers is entirely understandable and expected as the layers in a multi-layered defense will generally partially overlap each other... it's because of that overlap that the benefits of any individual layer may wane, but a layer isn't redundant or useless unless it's completely overlapped by the other layers...
so a question that is perhaps of interest to both posters is at what point does known-malware scanning become entirely redundant?... personally, i don't think it ever does... any given technology has an achilles heel that can allow a piece of malware to get past it... for known-malware scanning this achilles heel is novelty - new/unknown malware is clearly not something known-malware scanning can deal with... other technologies have their own weaknesses too but as weaknesses go novelty is unique because it's the only one that wears off... the media makes quite a big deal out of all the new malware being created and how it keeps getting past known-malware scanners, and at the same time it rarely mentions cases of malware incidents involving old malware... this leads to the erroneous impression that one needn't worry about old malware... old malware still pops up for a variety of different reasons and although it may use tricks that can still allow it to get past other technologies, by the time it gets old it's not going to be getting past a known-malware scanner anymore...
lonervamp also made a distinction between home use and business use, suggesting that known-malware scanning was more likely to be displaced in the business than in the home... i can certainly see that - one of the strengths of known-malware scanning is that the expertise that makes it work is embedded in the product so that the user is relieved of having that burden... many other technologies require knowledge and skill from the customer that may or may not be there... in a business you're much more likely to have trained, knowledgeable individuals (the IT staff) who can make the intelligent decisions necessary for products with more client side configuration to work effectively... that means that alternative technologies become a more feasible option in that environment...
that isn't to say i'd suggest dropping known-malware scanning though... even in a business i think it has it's place... i don't think it should be the only type of technology used, mind you - there are many types of technology available (sometimes even from a single vendor) and they're just waiting to be used... if i were going to suggest anything it would be more comprehensive layering, not simply different layers... the goal should be better overall coverage and substituting one layer for another isn't necessarily the way to get there...
Tags:
anti-malware,
anti-virus,
defense in depth,
HIPS,
lonervamp,
mcwresearch
Wednesday, February 06, 2008
samples, variants, and signatures - oh my
last week anton chuvakin characterized the new av-test.org results as "widely ridiculed"... this struck me as odd because i hadn't seen any ridicule so i asked for links... that didn't get me anywhere so i took to googling for the answer and trying to guess what keywords might match this nebulous ridicule anton was speaking of...
now, while i didn't find ridicule, per se, i did find some significant criticism in a thread at wilderssecurity.com by none other than paul wilders himself (i really need to find a way to aggregate web forums so i can actually follow these sorts of discussions instead of finding out about them after the fact)... the criticisms weren't totally unreasonable, in fact i even agree with one (that a good test needs thorough supporting documentation on things like test methodology, sample acquisition, test bed maintenance, etc - how i wish we could see documentation as thorough as the vtc used to put out) but some i think were a little off...
one such criticism had to do with comparing apples to oranges, or in this case the number of samples used in the test to the number of signatures in a product's database - which, when taken with some of the confusion that lead to, makes me think a discussion of what these things are might be in order...
a sample is just a file containing a piece of malware... you distinguish one sample from the next by simple binary comparison - if the files are different then the samples are different, otherwise they're just copies of the same sample...
a variant is a member of a malware family... one variant is distinguished from another by being sufficiently different from that other variant that it makes sense to consider it a different thing, but that is a subjective determination usually made in the context of what makes sense for a particular detection technology... there is no objective, technology-neutral standard by which we can say sample X represents a different variant than sample Y, therefore the distinction between variants is a technology-dependent logical construct... as an example, for a scanning engine that looks at every single byte of a piece of malware it would make sense to be a lot more granular in drawing distinctions between variants than for one that only looks at at most N bytes somewhere within the body of the malware... more generally, technologies that use fundamentally different approaches to malware recognition can likewise result in different delineations between variants... this is one of the reasons why they say each vendor counts viruses/malware differently...
signatures are chunks of data that tell a scanner what a particular piece of malware looks like... they can be a selection of bytes from the actual malware (ie. a scan string), a hash value (for non-parasitic, non-polymorphic malware), a regular expression, or something written in a proprietary virus(malware) description language... the conceptual ideal is that there is a 1-to-1 correlation between signatures and variants but in practice multiple signatures may be required for a single variant (as has sometimes been the case for handling polymorphism) or alternatively a single signature might handle multiple variants or even an entire family...
now obviously, given the above descriptions, comparing samples to signatures or variants doesn't make a lot of sense, but a variant-centric focus when evaluating a test of anti-malware products is not altogether unfamiliar... in a detection test it's important to make sure you include all (or as many as possible) significant members of the malware population and also to make sure that no member is included more than any other so that detection of each is given equal weight/importance... back in the days when viruses ruled the earth it was actually necessary to try and look at things from a variant-centric perspective (subjective as it was) because of how viruses worked... the way file infecting viruses operated, it was trivial to imagine many samples containing not just the same variant of a virus but instances that were all offspring of a common parent... this made it necessary to try to tag samples with variant id's in an effort to enforce good coverage and uniformity...
these days the kind of complications that parasitic self-replication create in malware detection tests are much less of an issue because malware exhibiting those properties make up a much smaller proportion of the malware population... malware that exists as a stand-alone file is now the norm so the simpler sample-centric approach (at least for the non-viral samples) becomes more reasonable...
one potential complication that might still make a variant-centric approach necessary is packed malware or more generally malware incorporating server-side polymorphism... at least that's an argument one might bring up (and i think packed malware was mentioned in the thread) but i think there's a stronger argument to be made for server-side polymorphism in the general case producing distinctly new variants in spite common root malware that they all came from... this is because the resulting malware is the product of both relatively old malware (though the root malware may not technically qualify as known) and unknown and unguessable transformations... likewise with packing in particular, not all scanning engines deal with repacked malware equally well (even though we know many packers, the packers themselves can be altered to produce something we couldn't have guessed) so arguing that many samples created from the same root piece of malware are all the same variant may not be reasonable in light of the subjectivity of the variant concept...
given all this, from the information available in the test results in question, there really doesn't seem to be anything all that wrong... that said, without the thorough supporting documentation, we can't really verify that things are all that right either so those who really can and do read more than just the executive summary are stuck taking the results of this test with a grain of salt (in addition to the grain of salt one takes any detection test in isolation with, since the relative effectiveness of various products can change pretty quickly over a short time)...
(and anton will just have to believe what he wants to believe about whether the results that seem worthy of ridicule point to a cherry-picked test bed or not - which would be some feat, considering it's size)
now, while i didn't find ridicule, per se, i did find some significant criticism in a thread at wilderssecurity.com by none other than paul wilders himself (i really need to find a way to aggregate web forums so i can actually follow these sorts of discussions instead of finding out about them after the fact)... the criticisms weren't totally unreasonable, in fact i even agree with one (that a good test needs thorough supporting documentation on things like test methodology, sample acquisition, test bed maintenance, etc - how i wish we could see documentation as thorough as the vtc used to put out) but some i think were a little off...
one such criticism had to do with comparing apples to oranges, or in this case the number of samples used in the test to the number of signatures in a product's database - which, when taken with some of the confusion that lead to, makes me think a discussion of what these things are might be in order...
a sample is just a file containing a piece of malware... you distinguish one sample from the next by simple binary comparison - if the files are different then the samples are different, otherwise they're just copies of the same sample...
a variant is a member of a malware family... one variant is distinguished from another by being sufficiently different from that other variant that it makes sense to consider it a different thing, but that is a subjective determination usually made in the context of what makes sense for a particular detection technology... there is no objective, technology-neutral standard by which we can say sample X represents a different variant than sample Y, therefore the distinction between variants is a technology-dependent logical construct... as an example, for a scanning engine that looks at every single byte of a piece of malware it would make sense to be a lot more granular in drawing distinctions between variants than for one that only looks at at most N bytes somewhere within the body of the malware... more generally, technologies that use fundamentally different approaches to malware recognition can likewise result in different delineations between variants... this is one of the reasons why they say each vendor counts viruses/malware differently...
signatures are chunks of data that tell a scanner what a particular piece of malware looks like... they can be a selection of bytes from the actual malware (ie. a scan string), a hash value (for non-parasitic, non-polymorphic malware), a regular expression, or something written in a proprietary virus(malware) description language... the conceptual ideal is that there is a 1-to-1 correlation between signatures and variants but in practice multiple signatures may be required for a single variant (as has sometimes been the case for handling polymorphism) or alternatively a single signature might handle multiple variants or even an entire family...
now obviously, given the above descriptions, comparing samples to signatures or variants doesn't make a lot of sense, but a variant-centric focus when evaluating a test of anti-malware products is not altogether unfamiliar... in a detection test it's important to make sure you include all (or as many as possible) significant members of the malware population and also to make sure that no member is included more than any other so that detection of each is given equal weight/importance... back in the days when viruses ruled the earth it was actually necessary to try and look at things from a variant-centric perspective (subjective as it was) because of how viruses worked... the way file infecting viruses operated, it was trivial to imagine many samples containing not just the same variant of a virus but instances that were all offspring of a common parent... this made it necessary to try to tag samples with variant id's in an effort to enforce good coverage and uniformity...
these days the kind of complications that parasitic self-replication create in malware detection tests are much less of an issue because malware exhibiting those properties make up a much smaller proportion of the malware population... malware that exists as a stand-alone file is now the norm so the simpler sample-centric approach (at least for the non-viral samples) becomes more reasonable...
one potential complication that might still make a variant-centric approach necessary is packed malware or more generally malware incorporating server-side polymorphism... at least that's an argument one might bring up (and i think packed malware was mentioned in the thread) but i think there's a stronger argument to be made for server-side polymorphism in the general case producing distinctly new variants in spite common root malware that they all came from... this is because the resulting malware is the product of both relatively old malware (though the root malware may not technically qualify as known) and unknown and unguessable transformations... likewise with packing in particular, not all scanning engines deal with repacked malware equally well (even though we know many packers, the packers themselves can be altered to produce something we couldn't have guessed) so arguing that many samples created from the same root piece of malware are all the same variant may not be reasonable in light of the subjectivity of the variant concept...
given all this, from the information available in the test results in question, there really doesn't seem to be anything all that wrong... that said, without the thorough supporting documentation, we can't really verify that things are all that right either so those who really can and do read more than just the executive summary are stuck taking the results of this test with a grain of salt (in addition to the grain of salt one takes any detection test in isolation with, since the relative effectiveness of various products can change pretty quickly over a short time)...
(and anton will just have to believe what he wants to believe about whether the results that seem worthy of ridicule point to a cherry-picked test bed or not - which would be some feat, considering it's size)
what is packed malware?
packed or repacked malware is malware that has been modified using a runtime compression (or encryption) program...
runtime compression programs compress an executable file and prepend or *append a stub to the file containing the code to decompress it at runtime...
malware authors (or even just people deploying malware) use this functionality as a cheap and easy way to turn a known piece of malware into something 'new' that no one has ever seen before and that known-malware scanners can't detect... this is one of the many possible ways to implement the transformation function of server-side polymorphism...
although the malware is transformed in a manner similar to how conventional polymorphic viruses transformed themselves, the generic solution that worked so well for polymorphism doesn't necessarily work as well for packed malware because the size of todays malware combined with the potential to chain multiple packing transformations together make the amount of work necessary to emulate the full unpacking operation much more expensive and time consuming in comparison... that said, dynamic translation promises speed improvements over conventional emulation so that generic unpacking may still be workable...alternatively, some vendors have taken to (or at least considered) identifying packers that are used primarily/exclusively for obfuscating malware and alerting on any file packed by such a packer...
*a less general approach, but one with a little more heuristic value is to identify those packers used primarily/exclusively for obfuscating malware and alert on any file packed by such a packer...
*the most straightforward approach, is of course to have a library of unpacking algorithms for known packers and use those to unpack the malware - however this packer-specific approach has many of the same (or at least analogous) limitations that known-malware scanning itself has as even minor variations in the packing/unpacking algorithm (which can easily be made for malicious purposes) can potentially result in a failure to detect which packer was used and/or unpack the malware properly...
[* corrections/clarifications prompted by vesselin bontchev]
back to index
runtime compression programs compress an executable file and prepend or *append a stub to the file containing the code to decompress it at runtime...
malware authors (or even just people deploying malware) use this functionality as a cheap and easy way to turn a known piece of malware into something 'new' that no one has ever seen before and that known-malware scanners can't detect... this is one of the many possible ways to implement the transformation function of server-side polymorphism...
although the malware is transformed in a manner similar to how conventional polymorphic viruses transformed themselves, the generic solution that worked so well for polymorphism doesn't necessarily work as well for packed malware because the size of todays malware combined with the potential to chain multiple packing transformations together make the amount of work necessary to emulate the full unpacking operation much more expensive and time consuming in comparison... that said, dynamic translation promises speed improvements over conventional emulation so that generic unpacking may still be workable...
*a less general approach, but one with a little more heuristic value is to identify those packers used primarily/exclusively for obfuscating malware and alert on any file packed by such a packer...
*the most straightforward approach, is of course to have a library of unpacking algorithms for known packers and use those to unpack the malware - however this packer-specific approach has many of the same (or at least analogous) limitations that known-malware scanning itself has as even minor variations in the packing/unpacking algorithm (which can easily be made for malicious purposes) can potentially result in a failure to detect which packer was used and/or unpack the malware properly...
[* corrections/clarifications prompted by vesselin bontchev]
back to index
Tags:
definition,
malware,
packed malware
Saturday, February 02, 2008
when is a PUP not a PUP?
when it hides itself...
i came across a post on the mcafee avert labs blog the other day talking about potentially unwanted programs adding 'rootkit' functionality and i was struck by the absurdity of it...
if a PUP is using stealth and taking steps to avoid detection, why are we bothering with the euphemistic classification of 'possibly unwanted program'?
that a program is possibly unwanted implies that there is also the possibly that it is wanted, but taking steps to avoid detection implies that the program's creators don't care if the user wants the program or not and are in fact anticipating that the user won't want it...
if the creators of the software don't genuinely believe people want their software, why are we giving them the benefit of the doubt? i can see some legitimate reasons for software to hide other things on a system (to protect them from uninformed alteration, for example) but i can't see any legitimate reason for a program to hide it's own presence...
i'm not going to call a spade a spade, i'm going to call it a god damned shovel - if what used to be called a potentially unwanted program has added stealthkit functionality then it's no longer a potentially unwanted program, it's fully fledged malware... not all spyware/adware/what-have-you qualified as potentially unwanted programs in the first place, it was always a very special and very tenuous distinction that applied only to the most benign instances... actively avoiding detection isn't the mark of something benign...
i came across a post on the mcafee avert labs blog the other day talking about potentially unwanted programs adding 'rootkit' functionality and i was struck by the absurdity of it...
if a PUP is using stealth and taking steps to avoid detection, why are we bothering with the euphemistic classification of 'possibly unwanted program'?
that a program is possibly unwanted implies that there is also the possibly that it is wanted, but taking steps to avoid detection implies that the program's creators don't care if the user wants the program or not and are in fact anticipating that the user won't want it...
if the creators of the software don't genuinely believe people want their software, why are we giving them the benefit of the doubt? i can see some legitimate reasons for software to hide other things on a system (to protect them from uninformed alteration, for example) but i can't see any legitimate reason for a program to hide it's own presence...
i'm not going to call a spade a spade, i'm going to call it a god damned shovel - if what used to be called a potentially unwanted program has added stealthkit functionality then it's no longer a potentially unwanted program, it's fully fledged malware... not all spyware/adware/what-have-you qualified as potentially unwanted programs in the first place, it was always a very special and very tenuous distinction that applied only to the most benign instances... actively avoiding detection isn't the mark of something benign...
Subscribe to:
Posts (Atom)