A funny thing happened on my way to some research on Anti-SPAM technologies, I found that Microsoft is wiping SenderID information from its website. When you try going to the web page for the SenderID Framework you are now redirected to a page for its Forefront anti-SPAM products. In fact, most links to SenderID information at microsoft.com now redirect to the Forefront page. The only exception seems to be some support articles and blog entries. And if you want some final proof, it looks to me like Microsoft is no longer publishing SenderID (i.e., SPF2.0) records in the DNS entry for microsoft.com! SenderID is well and truly dead.
SenderID was Microsoft’s proposal for eliminating spoofing in email, a major factor in phishing attempts. It was intended to make sure that mail claiming to be from someone at example.com really came from example.com. Unfortunately for Microsoft the rest of the industry pretty much rejected the SenderID proposal. A couple of alternate proposals have wider non-Microsoft support, Sender Policy Framework (SPF) and DomainKeys Identified Mail (DKIM). But neither is being adopted agressively enough to make a dent in the phishing problem.
SenderID seems to have died because the SPF and SenderID proponents couldn’t get together on merging the proposals. I can’t really tell if this was because of some Microsoft intransigence or the prevailing “anybody but Microsoft” (aka, ABM) winds of the time, and I don’t want to revisit the battle. The result was essentially a move by most of the Internet intelligentsia to support SPF while Microsoft futilly evangelized SenderID (supporting it both in Hotmail and in Microsoft Exchange). While SenderID adoption outside of Microsoft itself was very limited, SPF has achieved some measure of success. It appears that about 45% of larger domains, and 10% of all domains, publish SPF records. But sadly less than half of those do so in a way that will truly block forged email. Most users of SPF are merely using it to advise SPAM filters that an email might be suspicious rather than being clear that the mail did or did not originate from them.
Fidelity and Schwab? They specify the all important “-all” qualifier in their SPF record, meaning that unless the mail comes from one of their specifically identified servers it should be rejected as a forgery. Gmail, Hotmail, or Yahoo Mail? Nope, sorry. They have SPF records but instead of specifying that mail coming from a sever not under their control should be rejected they throw up their hands and basically say “we don’t know if that email really came from us”. Wimps. What about Ebay and Paypal, two of the most frequently spoofed domains? Nope, no -all. That doesn’t seem to make sense, does it?
Yahoo, Google, Ebay, and Paypal are putting more weight on the use of DKIM to address the forgery problem. Data on DKIM adoption is rather hard to come by, but if I boil down the bits and pieces I can find around the web then DKIM adoption amongst large ecommerce sites is high, on large sites in general it is about half that of SPF, and amongst smaller sites it may be non-existent. If we look outside the U.S., at Japan, then SPF adoption is approaching 42% while DKIM adoption is at 1/2%. The data makes it hard to tell how much success DKIM is really having.
The biggest problem with both SPF and DKIM aren’t their technical merits or flaws, it is that there isn’t clear industry acceptance and agreement to deploy one or both. In the case of DKIM I certainly consider Google and Yahoo’s support, along with major sites like Ebay and Paypal, a big plus. But while Gmail may reject forged mail claiming to be from Paypal, Microsoft’s Hotmail may not because Microsoft
doesn’t support DKIM and Paypal doesn’t have a hard-corps SPF policy has just started experimenting with the use of DKIM for mail that fails an SPF check. And since Paypal doesn’t specify -all (which would cause the SPF check for forged mail to fail) Hotmail users will have to hope that Microsoft’s filters find enough else wrong with the mail to classify it as a phishing attempt. Microsoft Exchange and Microsoft Forefront products also don’t have built-in DKIM support, so most corporate email servers will accept the forged Paypal email. I also doubt most ISPs support DKIM, for example from what I can find I don’t believe that Comcast or Verizon support it in their email systems.
With SenderID dead it certainly opens up the possibility that Microsoft will join the DKIM camp. Or at least evangelize SPF more aggressively.
If When Microsoft fully supports DKIM in Hotmail, adds native support in Exchange Server as well as Exchange Online (nee Office365), and in its Forefront anti-SPAM products, that should push the effort over the hump towards universality. In fact, wouldn’t it be grand if Microsoft, Yahoo, and Google declared a date that they would all drop any non-DKIM signed email on the floor (meaning they wouldn’t even put it in your junk folder, they wouldn’t deliver it at all). Failing that, an agreement that all email must be authenticated with either DKIM or SPF (including -all), with at least all the major email providers supporting both, would do. Now that would put a real dent in phishing.
I wanted to provide some clarity here. This link to Sender ID Framework SPF Record Wizard was effected during a rearrangement of content on that site. The direct link is http://www.microsoft.com/mscorp/safety/content/technologies/senderid/wizard/default.aspx and will shortly be updated on the AOTA site https://otalliance.org. It appears as well that major search engines seem to have the latest link already. Turns out that if you leave the “/default.aspx” off the link you get put to the anti-spam homepage. I want to thank you for the catch.
Microsoft continues to be committed to SenderID as well as DKIM. We find them to be complementary in many ways and current support both of them on inbound mail. SendierID and SPF currently have the largest adoption with our senders and thus we check them first. SendierID record have the added benefit of improved Phishing protection as we verify the address that is shown to the customer to match the sending domain. If this fails we then check against DKIM. Checking against DKIM helps us reduce false positives that may arise from email forwarding. Although adoption of DKIM has been slower due to the complexity of deployment for mailers we continue to urge senders to adopt both Sender ID and DKIM to insure the highest quality delivery as well as Phishing protection.
Microsoft General Manager – Safety Services
Thanks for the clarification John!
Your committed to DKIM, so when can we expect to see support for it in Exchange?
John’s not responsible for Exchange, or at least he had no relationship to it in the old organization structure.
Apparently, Hotmail is in the DKIM camp already. However, they only check DKIM for messages that fail SPF. See
Thanks for finding this! The article also states they are only applying this to 1.5% of inbound traffic, but the fact that they are in the midst of a rollout says a whole lot of good things!
Pingback: SenderID Followup | Hal's (Im)Perfect Vision
DKIM and SPF/SenderID aren’t in a battle for dominance, they’re complimentary in that they prevent different types of spoofing.
The adoption phases for each are likely to take a long time, and during those phase both carry risks of false positives – especially with forwarded email. The value of each technology increases for recipients as more senders adopt them, and for senders as more recipients check them.
Sender ID Frameworks works great! I try to encourage everybody to use it, if possible.
In fact, I don’t think it has to do with its design. What I notice it that a lot of companies are not aware of the Sender ID Framework. When I explain them, they are mostly excited. Why companies don’t implement it sooner, is because they help others with their own SPF record(s) and not themselves.
I do wish Microsoft would promote it better. Make people aware of its existence…