This is not a blog post for the IT professional, unless you have little idea how anti-malware, reputation, etc. work. It is a small primer for those who don’t quite get all this anti-malware stuff.
At its core anti-malware seems pretty straightforward. You take something that can be run (usually an “image” or “executable”, but for sake of the audience we’ll just say App) and run a mathematical algorithm over it that produces an (almost) unique identifier for that App. Generally that unique identifier is called a Hash (because of the technique used). Anytime you run the same algorithm over the same App you get the same Hash. Two different Apps with same name will have different Hashes. Two Apps that are identical but have different names will get the same Hash. If someone modifies an app, even slightly, it will have a different Hash. So the Hash tells you exactly what you have.
Anti-Malware software simply computes the Hash for an App and compares it to a “Black List” of Hashes that have been deemed Malware. Those updates your anti-Malware software does all the time are to update the Black List of Hashes (or Signatures) of known Malware. While this technique was sufficient in the early (particularly pre-Internet) days, today the Malware can spread rapidly enough to be endemic before your Anti-Malware vendor adds its Hash to their Black List. More recently Malware authors have taken to constant updating of the App with slight changes so that each copy has a different Hash, defeating the simple Black List mechanism.
With Black Lists of shrinking effectiveness Anti-Malware has grown to incorporate more sophisticated techniques for detecting Malware. The problem with these techniques is that they significantly slow down App execution, so another mechanism was needed to minimize their use. That is the addition of a White List of App Hashes that are known to be safe to run. Let’s say you go to run Microsoft Excel. The White List will contain the Hash of Excel’s main executable, excel.exe, as known to have been distributed by Microsoft. Anti-Malware just checks the Hash and lets Excel run with no further interference. If the Hash doesn’t match then either excel.exe has been modified or some other App has an executable called excel.exe. Either way, it won’t be mistaken for the real Microsoft Excel.
Most Apps you run will either be on the White List or Black List and thus Anti-Malware software either lets them run without further interference or blocks them. But what happens when an App is not on either list? That’s when the App will be deeply evaluated either before, or while, it is running.
The specific techniques for evaluating unknown Apps are beyond this blog posting but basically cover two broad areas. The first, known as Generic Signatures, is for the Anti-Malware software to scan the image for known indicators that this is a particular kind of Malware. This helps defend against the case where Malware authors simply keep issuing very slight modifications of their Apps as a way to avoid the Black List. When a new example of the Malware App is found its Hash is sent to the Anti-Malware vendor so it can be added to the Black List (and thus blocked with less performance impact).
The second set of techniques are more dynamic, monitoring an App for suspicious behavior. If suspicion that the App is Malware is high then execution is blocked, but if it is only slightly suspicious then it is allowed to proceed. Information about the App may then be sent to the Anti-Malware vendor for analysis and a final determination on if it should be added to the Black List or White List.
Since traditionally unidentified Apps that have either no or only mildly suspicious behaviors are allowed to run it is possible for Malware to slip through these evaluation techniques. To combat this more complete “Reputation” systems are being incorporated into Anti-Malware software as well as Operating Systems. With Reputation, an App is only allowed to run if it is on the White List. Unknown applications are completely blocked. The problem with this approach in the past has been that many legitimate applications would be blocked, making systems almost unusable. Recent advances have made Reputation far more practical, although in rare cases blocking of legitimate Apps will still occur.
Modern Reputation-based systems are more pro-active in adding Apps to the White List. In particular they can look at the (verifiable) identity of the author of the App and, if they have a reputation for producing safe apps, automatically add any new (or legitimately changed) App to the White List. Apps that engage in suspicious or clearly harmful behavior can be removed from the White List, blocking their execution. This will also “damage” the reputation of the App’s author, making it more difficult for their Apps to be added to the White List. Apps obtained from a tightly controlled “Store” like the Windows Store or Apple’s App Store receive the most direct reputation verification and are always on the White List (unless pulled from the Store). By changing the focus of Anti-Malware from the Black List to the White List Reputation-based systems protect against Malware even before it is identified as such.
Other techniques contribute to fighting Malware as well. URL Filtering uses a Black List/White List technique to keep users from navigating to web sites that are known to distribute Malware or have done so in the past. Email Filtering removes Malware-infested images from mail, and keeps SPAM (many of whose links take you to Malware distributing websites) out of your Inbox. Firewalls help block Malware from communicating to the outside world.
That’s the simple tutorial. Hopefully one of the things it does is explain why Reputation is so important. It’s really the only technique that gets ahead of the Malware authors.
Uhm… quite true, but what about the opposite approach ? I mean, the whitelisting one (as opposite to blacklisting) ?
I’m not sure what the question is. White listing is covered in the third paragraph from the bottom.
What I mean is that, while the “block all, only allow what I define” approach may not be well suited for home users, it IS there (and has been there for quite a while now) and, with a little of education if may help people staying safe; just in case, have a look here http://www.nsa.gov/ia/_files/os/win2k/Application_Whitelisting_Using_SRP.pdf and I think you’ll see what I mean
Ah, that kind of white list. I did start the article off saying it wasn’t for IT professionals, and I sure didn’t try to cover every aspect of fighting malware.
SRPs are a nearly unusable feature (and I have indeed run systems using them, see my posts about PredictableIT) that is incredibly labor intensive and error prone. They are best left to large organizations with IT staffs that can dedicate resources to creating and maintaining the policies. And that are trying to lock down systems for more than just security reasons. For example, why are your call center personnel installing and running the Times Reader? Or trying to install and play games? And it really really helps if you don’t value end-user satisfaction because your users will indeed find apps they have business justification for that are not on your white list. And they will be very unhappy with the beauracratic process necessary to add the app, even if they somehow manage to successfully get it approved.
Still, this is a culture thing. Organizations whose culture calls for extremely controlled PC environments can make SRPs work. Organizations without that culture would find SRPs unworkable, no matter how many resources they devote to it. For small organizations or individuals, SRPs might as well never have been invented.