the article in question is Do not believe everything you read but you should reflect on it - anti-virus software tests which was about (you guessed it) the consumer reports controversy... my comment was as follows:
"This is more scientific than the test method used by Independent Security Evaluatiors (ISE) used by Consumer Reports to conduct these tests on its behalft?"
yes, retrospective testing is more scientific than testing with lab-made viruses... retrospective testing uses viruses from the real world while ISE's test used viruses meant to approximate those from the real world (ie. it simulated the processes by which viruses are created in the real world based on a variety assumptions which may or may not be valid)...
"A 40% detection rate in a retrospective test is seen as being pretty good for most anti-virus software because it means that it detects all the new malware appearing during 3 months by heuristics and generic detections."
??? 40% detection rate means it detected 40% of the viruses, not all of them... it's considered pretty good because most do much worse than 40%...
"Using currently-known viruses to measure the performance of older AV engines is based on the assumption that the viruses we know about today will be the same kind as those that we will see in the near future. Hence, what was unkown three months ago should be representative of new viruses in general?"
the only assumption here is your own that retrospective test results are supposed to be generalizable... the reason retrospective tests (and other tests for that matter) are performed over and over again is precisely because they are not individually generalizable and a general sense of how well a product does can only be gained from looking at trends across multiple tests...
"Just for the record, zoo viruses are commonly defined as “existing only in lab environments” and thus not ITW (in the wild) until leaked. If they only exist in lab environments, what might their origin be?"
just for the record this is an oversimplification of what zoo viruses are... they do not exist only in labs, they are simply believed to not be actively spreading in the wild... they can still exist all over the place, in vendor labs, in private collections, or in the hot little hands of those who simply haven't gotten around to trying to spread them yet...
"Further, where is the line to be drawn at “vendor created”? Vendors get viruses in various ways but sometimes provide compensation to people providing them (e.g., also called vulnerability research). Examples of this can be found in the Perrun JPEG virus proof of concept and the Commwarrior.a SymbianOS virus."
if those are supposed to be examples of viruses whose creators were compensated by the vendors then i think you need to provide some proof... although a certain famous name in the av industry is believed to have paid for viruses, that was a long time ago and those efforts helped him become a pariah in the av industry and community...
"The AVIEN open letter:
>Public letter concerning the Writing of Viruses & How it Does Not Teach about Virus Prevention (May 30, 2003)
is not necessarily very helpful. I even wonder why some of these well known av industry representatives let their name stand on this letter."
consider reading the signatures more closely - they are not just av industry representatives, there are lots of people on the list that are outside of the av industry...
" > “…it is not necessary and it is not useful to write computer viruses to learn how to protect against them.”
One could interpret this statement as saying something similar to: trust us we know best and do not need to explain ourselves to people like you."
or you could interpret it as simply the result of a great deal of online discussion that occurred elsewhere (in multiple venues)... petitions generally don't include all the conversations and arguments that lead up to their creation...
and that pretty much covers all the corrections i have saved up... for now...
0 comments:
Post a Comment