i wandered lonely as a cloud

relax, i'm not about to start waxing poetic about daffodils. rather i'm thinking about cloud-based anti-malware software.

it's something i've been thinking about for a little while now but i've finally decided to commit my thoughts to a more permanent format and share them with others.

for the past couple of years the major anti-malware vendors have been deploying cloud technology to improve the effectiveness of their products. often this has been an optimization specifically for their known malware scanners, although some have also taken the opportunity to build reputation systems.

it occurred to me that the cloud could be used for a great deal more than just that. think about what those reputation systems are doing. the user is faced with a complex question - is file X safe - and the cloud answers. the cloud can do this either because there are experts feeding the cloud it's answers or because there's a community feeding the cloud it's answers (or both, come to think of it). the point is that the cloud reduced the complexity for the user.

now think for a moment about all those technologies that have sprung up and then fallen by the wayside over the years. how many of them fell out of favour because they required too much knowledge, because they asked too much of the user? do you see where i'm heading yet? the cloud as a complexity reducing technology (alright it technically transfers and collates that complexity, but from the user's perspective it reduces it) seems like it actually has the potential to breathe new life in virtually all of those other techniques, be they sandboxing, whitelisting, behaviour blocking, or even integrity checking.

and of course, as i was originally coming up with that list i was reminded of the fact that many of them have actually been augmented with some kind of cloud technology to help take the complexity out of their operation. those efforts simply haven't been particularly mainstream. the biggest vendors have been slow to recognize the opportunity to augment these technologies (which can be superior in the right hands) with complexity reduction as a service. the smaller vendors that are taking a chance with this don't necessarily have the stability to keep it going. it would be nice if those other options saw more more mainstream deployment and adoption.

privacy implications of cloud-based security

in my continuing efforts to get caught up on my rss feeds (i'll get there eventually) i came across an interesting post at the stopbadware blog about establishing expectations for av vendors. it raises some interesting concerns about data collection and lack of transparency / informed consent.

this is a tough call. recall that when trend bought hijack this and added a button to send the log to trend for analysis people went ape over the idea that it was sending data without informing the user (even though the behaviour seemed clear to me just by looking at what the UI stated).

all cloud-based security inherently is going to collect data and send it into the cloud - that's how you leverage the cloud for security. clearly there is the potential for a privacy violation if the vendor isn't careful and there is certainly room for people to assume there already is a privacy violation. some people are inevitably going to cry foul if you tell them that their security software is sending data into the cloud.

and yet on the other hand if you don't tell them then they can't make intelligent, informed decisions about whether they want to accept the risk associated with cloud-based security technologies and so may fall into a false sense of privacy (not unlike a false sense of security).

i'm a big fan of informed decisions - i think that given enough information, reasonable people will be able to make intelligent decisions and hopefully that works out in the vendor's favour (if they're doing enough to protect customer's privacy).

i think security vendors absolutely need to inform their users, and i think the failure to do so should be considered a badware sort of behaviour. i think the risk of backlash can be mostly mitigated by informing the user HOW their privacy is being protected in spite of the data collection (ie. exactly what data is being collected and how the data is being anonymized - maybe even let them audit the data being sent). assuming the vendor is doing an adequate job of protecting users' privacy, only unreasonable people should continue to have a problem in the face of such transparency - and there isn't really much you can do for unreasonable people. all cloud-based technologies involve data collection, so those unreasonable folks will simply have to learn to be reasonable or seek out one of the dwindling number of products that don't have any cloud-based components.

the benefits of scanning in the cloud

now i know a good chunk of the general security industry has been poo-pooing the cloud recently, and normally av is the security industry's favourite whipping boy, so maybe this is just a case of two bad tastes that taste bad together... that being said, the concept has significant promise to take back several tactical advantages that av hasn't had in, well, forever...

1. signature generation to client update time is reduced/minimized
   
   usually a good thing and this time it isn't at the expense of q/a because rather than cutting a corner that affects quality they're cutting the corner of updating the client in the first place... instead they'll be updating the cloud which they have direct control over (unlike the client) and then the client doesn't need to detect that it's out of date and try to update itself (and hope updating hasn't foolishly been disabled)... this potentially gives vendors time to do more q/a on their signatures and so reduce the bad signature release rate (though it's already pretty low)...


2. under-reporting of new samples is reduced/minimized
   
   under-reporting is the single biggest advantage that targeted attacks have... without the law of large numbers favouring someone noticing there's something fishy about a particular file, that file doesn't get submitted for analysis and nobody gets signature-based detection capabilities for it... in cloud-based scanning, however, just about everything (except those things the user feels are too sensitive to transmit, and as such are probably not malware) should get submitted to the cloud so that is no longer an issue...


3. greater situational awareness/intelligence than ever before
   
   data on detections can be correlated and analyzed, etc. providing the potential for virtually every client to become a sensor in a giant honeynet... geographic and demographic trends/patterns in the attacks have the potential to be more easily seen with so much more real-world data, and those are things that can be used to better predict who's at increased risk or maybe even help to pinpoint the source(s) of the attacks...


4. conventional malware q/a should be entirely thwarted
   
   the ease with which current malware evades known-malware scanning is on the verge of becoming history... the basic methodology for evasion is to iteratively produce samples and run them past a slew of scanners to see if it's different enough to avoid them all (or enough of them to be valuable)... this worked in the past because malware authors could do this without anti-malware vendors being any the wiser... with a cloud-based scanner you can't fully scan a new malware sample without the vendor getting a copy (either the sample is submitted to the servers controlled by the vendor, or the sample is not submitted and the malware author gets incomplete/inaccurate detectability results) and thus letting the cat out of the bag about not only what your new malware looks like but possibly also what the heck you're up to ("hello, police, i'd like to report a large number of new malicious programs being generated at the following IP address")...


5. scanner reverse engineering is almost completely nullified
   
   before what we now know of as malware q/a existed, the more clever malware authors were believed to have reverse engineered various scanners looking for information they could use to make sure their malware would better avoid detection... and even today, those who deal in vulnerabilities (either for the betterment of security or for malicious gain) will analyze scanners looking for flaws that an attacker could take advantage of... with the scanning engine no longer residing on the client computer the only kind of analysis anyone without source code access can do is black-box analysis (and if a botnet can detect attacks and protect itself a cloud-based scanner should conceivably be able to as well)... in this way the scanning algorithm becomes as inscrutable as any server-side polymorphic engine...

countering malware quality assurance

just a quick post to point out something i just realized - maybe it's obvious to others, maybe not...

i was reading dancho danchev's umpteenth post on malware q/a when it struck me that the recent trend by vendors to put the scanning engine in the cloud effectively kills malware q/a... i suggested before that randomizing heuristic parameters might combat it, but that's probabilistic and comes at the cost of false positives... cloud-based scanning on the other hand ensures that the scanner implementing this new architecture cannot be used effectively (if at all) in traditional malware q/a because the samples will either be given to a server that the av vendor controls (thus destroying the samples' value to an attacker), or if the malware tester manages to sever the ties with the av server then the testing will give an incomplete and misleading result regarding the detectability of the malware in question...

each new scanner that goes this route is another scanner removed from the pool of scanners that malware q/a testers can use and with symantec, mcafee, trend, and panda (and perhaps more that i can't think of at the moment) having already gone this route that's a significant portion of the av user-base which will soon no longer be at the mercy of malware q/a...

i have no idea if this was intended or serendipitous, but either way it's still a good thing - and once again it proves the point that for every measure there exists a countermeasure...