RE: [funsec] Consumer Reports Slammed for Creating 'Test' Viruses

[Larry Seltzer]

>> There is a more scientific way of measuring real proactive detection
of AV products on future malware - it is called "proactive testing" or
"retrospective testing". The idea is to measure, say, 3-month old AV
product against real field viruses that appeared within these last 3
months.

I think "retrospective" is the apt term; "proactive" doesn't fit the
definition. This tells you how good your product was 3 months ago. I do
agree it gives you a better picture of how good your product was at that
point than testing fake viruses does today, but clearly it's not the
same thing. It also requires you to collect a large and representative
sample of malware, which can be hard to do if you're not in the business
full-time.

I've been in the position of testing heuristic AV protection and what CR
did is very tempting. I considered it and was talked out of it. The
alternatives weren't very good.

Larry Seltzer
eWEEK.com Security Center Editor
http://security.eweek.com/
http://blog.eweek.com/blogs/larry%5Fseltzer/
Contributing Editor, PC Magazine
larryseltzer at ziffdavis.com