Supporting other platforms before Windows

A few Microsoft properties have received grief the last couple of years about shipping features, or even entire apps, on non-Microsoft platforms before those same features or apps come to Windows and Windows Phone.  I talked to a friend about this a few months ago,  and as rumors swirl that Office for iPad may arrive before a Metro/Modern version of Office for Windows 8.x I thought I’d relay his explanation.

What groups inside Microsoft are finding, just as third-party developers have found, is that the API set in WinRT and on Windows Phone is deficient compared to Android and IOS.  So the development team envisions a feature they want to add.  It takes them a couple of days to implement that feature for Android or IOS.  But for Windows/Windows Phone they get into a cycle of negotiating a feature request with the OS team and then waiting for an OS update that includes the feature.  That can take man-weeks of effort and many months of elapsed time.

Now the app or services team faces a dilemma.  They can wait the many months for the Windows support to appear while they lose competitive ground, or they can ship their feature on Android and IOS as soon as their own update schedule allows and play catch-up on Windows.  Years ago they most likely would have taken the hit to their own business in order to protect the Windows franchise.  However in an age where Microsoft is an underdog in many areas that is no longer considered a viable way to do business.  Thus we will sometimes see features or entire apps on non-Microsoft platforms before we see them on Windows/Windows Phone.

Now of course this really should be putting pressure on the OS team to expose a greater and more competitive set of features through their modern API sets.  This is something third-party app developers are getting rather vocal about as well.  So on one hand a lot of Microsoft fans are going to get upset as functionality comes to Android and IOS before appearing on the various flavors of Windows.  On the other, they should be happy that Microsoft teams are putting a lot of internal pressure on the OS team that in the medium to long-term will greatly improve Windows as a modern app platform.

About these ads
Posted in Computer and Internet, Microsoft, Mobile, Windows, Windows Phone | Tagged , , , , , | 16 Comments

A call for EMET Lite

Often I make suggestions to Microsoft privately, occasionally I do so publicly.  I’m doing this one publicly to generate broader discussion and hopefully a consensus.  I already mentioned this on Twitter a few weeks ago, but the full discussion requires a blog entry.

Microsoft’s EMET (the Enhanced Mitigation Experience Toolkit) is a security tool aimed primarily at Enterprise Information Technology departments.  It can be used by, and is available to, sophisticated end users.  However it really isn’t designed for typical end-user use.  This is a call for Microsoft to create an “EMET Lite” that is available with or packaged into all Windows systems, with management provided by Microsoft via Windows Update.

To get an idea of why EMET Lite might be desirable take a look at the results from this week’s Pwn2Own hacking contest.  No one was able to claim the $150,000 Grand Prize for hacking IE11 with EMET running.  All major browsers, including IE11 without EMET, were hacked.

Ok, so what is EMET?  Let’s go back to the effort that Microsoft started in the early days of Windows XP when it became apparent that the OS had severe security problems in the Internet environment.  It started to add features (DEP, SEHOP, ASLR, etc.) to the operating system that applications could use to harden themselves against attacks.  Why did applications have to explicitly turn those features on instead of the OS just imposing them?  Easy, in many cases applications required minor changes to be compatible with the new security features.  So the model from Windows XP SP2 on has been that executables have to indicate when built that the features should be turned on.

Now Microsoft itself made turning on those features part of the Security Development Lifecycle (SDL) for its products, so those are fairly well protected.  And over the years many other software developers have adopted SDL or similar processes and turned on these features.  But what about applications that haven’t turned on those features?  What about bespoke applications that an IT shop writes that are no longer being actively developed?  What about apps that the source code is unavailable for?  Tweaking these apps and rebuilding them to use the features runs from impractical to impossible.  The answer to that problem was EMET.

EMET allows an IT shop to force one or more security features on for a particular executable.  So let’s take an example of how it was intended to be used.  You have a bespoke order processing application, and you have some kind of internal testing methodology for verifying changes to that application.  So you take EMET and you use it to turn on one or more of the features.  Then you test to see if you’ve broken the app.  If you haven’t broken the app then you deploy an EMET rule to all your clients turning on the features(s) for the order processing app.

The key here is that the IT department is responsible for testing and making sure the app is compatible with the selected security features.  And if the app is updated, the IT department is responsible for re-testing that the features don’t break it.  These are things beyond Microsoft’s control, and beyond what 99.99% of end-users are willing to deal with.  That’s why EMET is a toolkit and not simply an OS feature.

But if Microsoft is already mitigating its own software, and so are many ISVs, then isn’t EMET essentially only for bespoke apps?  Well, no.

Microsoft keeps expanding the set of mitigations available through EMET, with mitigations appearing in it before they are available through the OS and development tools.  Moreover, even if a new mitigation were available and used to protect “IE12″ that wouldn’t help IE11.  So EMET can be used to add newer mitigation techniques to current, or older, software releases.

This is great for IT shops who can, and should, be using EMET to protect all software running on systems they are responsible for.  But what about the rest of us?

I propose that Microsoft create an EMET Lite that is distributed to users much as Microsoft Security Essentials and Windows Defender are today.  That is, either a free and recommended download or built-in to newer versions of the operating system.  The key differentiator between EMET and EMET Lite is that for the latter all of the rules would be generated by Microsoft and managed via Windows Update.    This places a burden on Microsoft, which is likely why they haven’t done it to date.  But for a company worried enough about security that they created EMET, and with evidence of the value of an EMET Lite such as the Pwn2Own results, Microsoft should take on this burden.

How much of a burden would Microsoft managing the EMET Lite rules actually be?  I don’t think it would be substantial.  Take as an example a default set of rules that come with the EMET 5 Technical Preview.  They turn on mitigations for “Microsoft Internet Explorer, WordPad, applications that are part of the Microsoft Office suite, Adobe Acrobat 8-11, Adobe Reader 8-11, and Oracle Java 6 and 7.”  So if you install EMET and accept the defaults you already have protected critical software using Microsoft supplied rules.  Now all they need to do is offer to update those rules as needed with Windows Update and you are rather close to my EMET Lite offering.

EMET Lite could be offered in a way that was almost totally transparent to end-users.  It could be distributed via Windows Update as a recommended download (and built in to post Update 1 versions of Windows 8.1 and later).  Once downloaded Windows Update would maintain the rule-set.  Telemetry from application crashes, as well as Microsoft’s other feedback loops, would be used to fix broken rules.  The testing processes used for Anti-Malware signatures and Patch Tuesday updates could be applied to proposed rule changes.

Third parties could be encouraged to validate and supply rules for their own software that Microsoft would then ship, though this carries some complexity and risks.  It seems that Microsoft already has many cooperation frameworks which could be extended to cover this case.  If not, Microsoft might simply let third-parties install and maintain their own rules.

EMET Lite also offers Microsoft an additional way to deal with some zero-day issues while it, or an ISV, develop a patch.  It could ship a new rule, or create a “Fix It” solution that installs a new rule, turning on a mitigation even if that creates a compatibility problem pending a real fix.  The Fix It path is particularly attractive because it allows Microsoft Customer Support to help customers while engineering is still investigating a permanent solution.

The benefits of EMET Lite seem enormous, the downside minimal.  Microsoft would take on some extra costs and risk.  But those costs and risk seem pretty minimal compared to the benefit that EMET is demonstrating.  Now is the time for EMET to move from IT toolkit to mass market security tool via an EMET Lite.

Posted in Computer and Internet, Microsoft, Security, Windows | Tagged , , , | 7 Comments

NEW post on Windows XP situation

I made two incorrect statements in my last post on Windows XP.  One is that I said it was my last warning on its demise, and obviously here is another one.  The other is that I said I’d write about how to live with Windows XP after support ends on April 8th, and I haven’t.  That’s the result of this blog being purely a hobby and having lost interest in the topic for a couple of months.  So this is an update on the Windows XP situation with one month to go before Microsoft ends support.

First a little story.  A couple of weeks ago I walked into the office of a new consulting client and staring me in the face was a Windows XP system.  I mentioned the coming end of support to the COO and a look of concern crossed her face.  She asked that I mention it to the VP of Administration, who owns IT, which I did.  She had the situation well in hand, with only 3 or 4 of their machines not yet moved off XP.  And they likely will be by next month.  So yes, a lot of people may have their head in the sand and be surprised when Windows XP support ends.  But it may be far fewer than most of us have been worried about.

Getting away from the anecdotal evidence let’s look at some numbers.  The panic-level numbers that have been in the press lately are global numbers from Netapplication.  As I’ve pointed out before, I can’t drill into those without paying so I’m going to use Statcounter numbers instead.  Statcounter paints a little rosier picture than Netapplication, with Windows XP continuing to be used by 18.6% of desktop computers on a worldwide basis as of February 2014.  Sounds a lot better than the 29% that has been in the press, right?  It is still too high though.  And the methodology of both organizations have their flaws, but they both provide data that is useful.  So let us drill in.

The truth about Windows XP usage is that looking at the situation globally gives a very distorted picture.  Why?  Well in China, where piracy was (and is) rampant Windows XP remains the leading operating system with 48.26% of the desktop OS market!  This has a number of implications which I’ll get to in a moment, but the first is that Chinese usage of Windows XP is really distorting the global number.

Here in the United States Windows XP usage is reported as 10.93% by Statcounter.  To put that in perspective, they report Mac OS X (all versions) usage as 18.07%.  Since malware authors target large populations, and historically OS X has been targeted less because of relatively low usage compared to Windows, this suggests malware authors may actually start losing interest in Windows XP!

It may not happen immediately, but if Windows XP share continues to drift down over the year after support ends then not a lot of economically-driven hackers aren’t going to be wasting their time searching for new XP-specific vulnerabilities.  Android, for example, has become a much juicier target.  The exception will be those in the “Advanced Persistent Threat” world, where you might be looking to launch a targeted attack against an entity you know is still running XP.  Think a Stuxnet-type attack.  This is something the corporate and government worlds need to take very seriously, and continue to push to eliminate XP from their operations.

In any case, 10.69% is a lot better than the 29% headline number.  In Australia XP usage is already down to 7.62%, and I imagine the U.S. will be there within a year.  This starts to get us down into the noise range, at which point you basically declare mission accomplished.  Europe is at 16.48%, which is surprisingly high.  But individual countries are all over the map.  The United Kingdom is at 8.53% while Poland is at 25%.  As a general rule North and South America, Europe, and Oceana are below average while Asia and Africa are above average.

Now there are probably some people who are happy with half of computers in China still running Windows XP.  The NSA is one of them.  But on an overall basis this is a very disturbing situation.  Western companies do a lot of business with China, and will now be sharing confidential information with entities running vulnerable systems.  It also blunts my argument about economically focused hackers losing interest in XP.  So with China, as well as other lingering high-usage countries, Microsoft and its ecosystem must retain their focus on migrating users off Windows XP.

Why is the situation in China so bad?  I can think of two reasons.  One of those is the high degree of software piracy in that country and the difficulties in engaging with owners of pirated software.  Second may be the economic reality of a much higher percentage of systems not being capable of running Windows 7 and later combined with an inability to afford a replacement system.  Similar factors may be impacting India (28.97%) and a number of other countries.

So what does all this mean?  I’m not sure.  In countries where Windows XP usage has dropped below 10% the situation moves from apocalyptic to problematic.  But on a worldwide basis, with a global connected economy, the problem is as bad as ever.  And it seems like no amount of effort by Microsoft, or other organizations, may drive down XP usage in places like China.  Not even the end of support.

Posted in Computer and Internet, Microsoft, Windows | Tagged , , | 8 Comments

Finishing turning Windows Blue

People seem a little shocked at how much change is coming in both the Windows 8.1 Update (a.k.a. Update 1, Feature Pack, etc.) and in Windows Phone 8.1.  You probably shouldn’t be as both offerings have far more effort behind them than their names suggest.   Recall that all of this is part of a product “wave” Microsoft referred to internally as Blue that entered planning as Windows 8/Windows Phone 8 were being finalized .

Back before Windows 8 shipped I wrote a blog entry on how Microsoft might adapt the Windows engineering system to deal with quick turnaround releases.  One alternative I proposed is that they would do a planning phase that covered multiple releases rather than separate planning phases for each release.  That’s exactly what I think we are seeing with the Windows 8.1 Update.

Microsoft did a planning exercise for Windows 8.1, saw how many development milestones they could fit in to make the 8.1 RTM target, allocated work to those development milestones, then created a post-8.1 development milestone and allocated the remaining 8.1 work to it.  That added development milestone is what we now see as the 8.1 Update.  But the plan and most of the decisions and designs were done more than a year ago as part of the original 8.1 effort.  Even a lot of the development work behind the Update may have been done as part of an earlier 8.1 milestone but it couldn’t be completed until the post-RTM milestone.  At least I believe that’s the case.

So whereas most of us see Windows 8.1 as what Windows 8.0 should have been, the Windows 8.1 Update is what Microsoft envisioned Windows 8.1 being!

Meanwhile Windows Phone 8.1 is an even bigger deal, and would no doubt have a more impressive name (8.5 or 9) if it weren’t for the attempt to line up Windows and Windows Phone version naming.  Well, it isn’t just naming it is an attempt to coordinate the releases much more closely, something that really kicks into gear for the post-Blue product wave after the One Microsoft reorg.

Consider that Windows Phone 8.1 is the first release of WP where the development team has gotten to place its primary focus on features and functions for customers and OEMs.  It is also the first release since the original Windows Phone 7 effort to have an 18 month (rather than 12 month) release cycle.  There is some amount of arbitrariness in picking how long the Windows Phone 7 cycle was, but basically it is 18-24 months depending on your perspective.

A year after Windows Phone 7 we had 7.5, which matured the platform a bit.  But even as Microsoft worked on that release it was ramping up the activities to move to a completely different kernel.  And then the entire team was focused on that kernel move for Windows Phone 8.  That meant for Windows Phone 7.5 and Windows Phone 8 we had modest (but important) improvements for users and developers, but most of Microsoft’s effort was going on under the covers.

Windows Phone 8.1 represents the first release of Windows Phone where the majority of the team doesn’t have to be focused on a new kernel.  Sure there is kernel-related work going on, like support for lower-end chipsets, but the teams working on higher-level layers finally had a level of stability in the lower-level layers that should have let them focus most of their resources on user visible improvements.  Combine that with the longer 18 month development cycle and Windows Phone 8.1 had better be a major step forward in user visible functionality!

Let me state this more strongly.  I would expect Windows Phone 8.1 to be the release where Windows Phone stops being an OS that is clearly less mature than IOS and Android and instead enters a phase where it is playing the same leapfrog game that the two of them have been engaged in.  From the leaks it seems that’s about where 8.1 is going to land.

Do the Windows 8.1 Update and Windows Phone 8.1 represent the end of the rollout of the Blue wave of OS changes?  Probably, with the caveat that Microsoft might release some minor to modest updates (similar to the Windows Phone 8 GDRs) before what is probably a Windows/Windows Phone 9 release.  Hopefully we’ll get some idea of their strategy, if not a full (PDC-style) developer-oriented reveal, at Build 2014 in less than a month.

 

 

Posted in Computer and Internet, Microsoft, Mobile, Windows, Windows Phone | Tagged , , , , , | 8 Comments

SQL Server 2014 Delayed Durability/Lazy Commit

I am having a lot of fun watching everyone get excited over SQL Server 2014’s Delayed Durability feature, mostly because I invented it back in 1986.  At the time no one was particularly excited by the idea.  It’s possible someone invented it before me, but I never found any evidence of that.

Not long after taking over as Project Leader for DEC’s Rdb product I was looking at ways to address one of its performance bottlenecks, the wait to flush the log on a commit.  For those not schooled in database technology a key part of ensuring no data is lost on a system failure (a.k.a., durability) is to require that changes be written (forcibly if necessary) to the log file before you acknowledge a transaction has been committed.  The log file is sequential, so writing to it is enormously faster than writing changes back to the database itself.  But you still have to wait for the write to complete.  The architecture of Rdb 1.x and 2.x did not allow for what is now known as Group Commit or a number of other techniques for speeding up commit.  Further each database connection had its own log, so that even log writing typically required a seek (i.e., it was still random rather than serial) thus limiting throughput and typically imposing a 100+ms delay to commit.  On heavily loaded systems I remember this climbing to 250ms or more.  Since we couldn’t implement Group Commit in a minor release, I was thinking about other answers and had a revelation.

For many applications a business transaction (e.g., add a customer) is actually a series of database transactions.  From the application perspective, the customer add is not complete until the final commit of the series of database transactions, and thus they already have (or could easily be written to have) recovery logic that deals with failures (e.g., system crashes) between those individual database transactions.  In effect, the durability of those individual database transactions was optional, until the final one that completed the business transaction.

With this in mind I went and prototyped Delayed Durability as an option for Rdb transactions.  On Rdb it was quite simple to implement and I literally had it working in one evening.  But these were short turn-around releases, I was treading into another team’s area (the KODA storage engine), and there just wasn’t time to finish productizing it.  So a couple of weeks later I pulled out the change and in Rdb 3.x we started working on other (app transparent) solutions to the synchronous commit performance problem.

Now jump forward to 1994 after I’ve joined Microsoft.  There is somewhat of a battle going on between the team working on the first version of Microsoft Exchange (nee 4.0) and the JET Blue storage engine team over performance issues Exchange was having.  Because I was new to Microsoft and had no biases I was asked to look into Exchange’s performance problems.  That was quite the experience but I’ll limit this to just the relevant story.  I learned that to send an email the client did a series (30-40 pops into mind as typical) of MAPI property set calls.  And each one of those calls was turned into an individual database transaction.  Which of course meant 30-40 synchronous log flushes per email message!  No wonder they were having significant performance problems.  While my major recommendation was that they find a way to group those property sets into a single transaction, I had another trick up my sleeve.

After confirming that Exchange was fully designed to handle system failures between those MAPI Property Set “transactions” I suggested to Ian Jose, the Development Lead for JET Blue, that he implement Delayed (I think I called it Deferred at the time) Durability.  The next day he told me they’d added it, and so to the best of my knowledge the first product to ship with Delayed Durability was Exchange 4.0 in April 1996.  A full decade after I first proposed the idea.  Of course that wasn’t visible to anyone except through the greatly improved performance it provided.  But still I was quite proud to see my almost trivial little child born.

With SQL Server 2014 shipping Delayed Durability as a user visible feature my little child is finally reaching maturity.  It only took 28 years.

Update: My friend Yuri pointed out that Oracle implemented Asynchronous Commit in Oracle 10gR2 in 2006.  So it only took 20 years, not 28,  from my invention until the feature appeared in a commercial product.

Posted in Computer and Internet, Database, Microsoft, SQL Server | Tagged , , , , | 22 Comments

Satya shuffles his leadership

Now that Microsoft has formally announced a bunch of Senior Leadership Team changes I thought it appropriate to comment.

The easiest one is Tony Bates.  Like many CEOs who join a large company as part of an acquisition it can be hard to find an appropriate role in said large company.  I assumed that Tony took the Business Development job as a landing spot until something more appropriate came along (which could have been Microsoft CEO or ownership of a future business).  With Satya’s ascension to CEO Tony probably concluded there was nothing appropriate for him in the reasonable future.  In fact the most appropriate roles probably required that he, like Julie Larson-Green, would have had to give up his EVP position for a CVP position.

Speaking of Julie Larson-Green, with the closing of the Nokia deal she wasn’t going to be heading the Devices business at Microsoft.  So, if you have to play second fiddle you might as well choose a role that you have real passion for and unique skills in.  Julie’s new role working for Qi seems to fit the bill.  Good move for Julie and Microsoft.

Tami Reller’s departure is a bit of a surprise, but not a huge one.  I did not expect Satya’s ascension to CEO to necessarily impact Tami because they’ve worked together before.  They certainly were peers in Microsoft Business Solutions, and Tami may have briefly reported to Satya (though I can’t recall all the timing).  As far as I know they had a good working relationship.  But Tami’s background was finance and her marketing chops relatively limited.  So owning all of marketing for Microsoft was clearly a stretch.  Whatever discussions have happened over the last month it must have been clear to Tami, Satya, or both that Tami wasn’t the person for the job the way that Satya saw that  job.  And as much as I respect Tami, I don’t see her leaving as an earth-shattering departure.

Meanwhile, combining Tami’s responsibilities with the advertising responsibilities that Mark Penn had and giving them to Chris Caposella as Chief Marketing Officer is a great move.  Chris is probably the most respected marketing leader inside Microsoft.  Recall that Chris actually was the Chief Marketing Officer, taking over Central Marketing (as well as creating the Consumer Channels Group), from Mich Mathews 3 years ago after a long spell as the head of marketing for all Information Worker oriented products.  Along the way his CMG responsibilities moved to others and he was left with CCG.  This reorg not only reinstates his CMO title and CMG responsibilities (which included corporate advertising before Mark Penn took them over) but gives him leadership of all Microsoft product marketing as well.  Chris actually was the most logical choice to have been given this role as part of last summer’s One Microsoft reorg, but lost out to Tami  in the game of musical chairs.  Now balance has been restored.

Which brings us to Mark Penn.  I don’t know him and I really don’t know how insiders think of him.  Outsiders seem to base their opinions on their like or dislike of the Scroogled advertising campaign.  Of course I expect that campaign represented about 1% of his efforts since joining Microsoft and is almost irrelevant in the greater scheme of things.  In his revised role it looks like he lost direct operating responsibilities but retained his advisory role on corporate strategy.  He even had his title altered slightly to reflect the new focus (and likely to sooth his ego over losing control of advertising).  He is still EVP of Strategy, but also is called out as Chief Strategy Officer.  Microsoft has been throwing “Chief” around a lot the last few years and I don’t associate any incremental influence or power with those titles.  What is important is what the actual job entails.  Read Satya’s mail I linked to above for a good description of that.

That just leaves Eric Rudder taking on Tony Bates’ responsibilities on an interim basis.  I think the interim part is real and this was just a move to avoid overloading Satya with one or more additional, and perhaps junior, direct reports while he searches for a new leader for this function.  Or decides to organize it differently.

I don’t think any of these moves are earth shattering in the short run.  They do represent incremental changes that will lead to a more cohesive Senior Leadership Team.  They also mean that the SLT is more heavily weighted with people who grew up with the company and were (often as junior individual contributors) part of its glory days.  They no doubt are highly motivated to be known as the ones who returned the company to unquestioned leadership status.  I can’t give an unbiased opinion on if this is the right direction to take (my biased one is yes), but there is one thing I’m sure of.  Steve Ballmer spent a lot of time flailing about looking for a formula, including hiring many outside executives at the Senior Leadership Team level, to propel Microsoft forward.  With Satya’s appointment as CEO, and his management moves so far, it is clear that the primary bet is on home grown leadership.  That could reignite a lot of passion within the company.

Posted in Computer and Internet, Microsoft | Tagged , , , , , | 17 Comments

Another stupid Anti-Malware test?

PC Mag has an article on the latest anti-malware tests by NSS Labs.  For this test NSS Labs turned off Microsoft’s SmartScreen because it was too effective, blocking 98-100% of malware.  SmartScreen, being in the OS now, works independent of browser or anti-malware software installed.  So NSS Labs decided to turn it off so they could test the Anti-Malware products alone.

Ok, but there is a BIG BUT here.  3rd Party anti-malware products come with their own URL filtering and whitelisting capabilities.  Microsoft considers SmartScreen part of their overall protection suite and has no reason to duplicate its functionality in MSE/Defender.  So turning off SmartScreen is the same as going into a 3rd party product and disabling its similar features.

Does NSS Labs disable the SmartScreen-equivalent features in the other anti-malware products it tests?  I don’t think so.  The only anti-malware that is tested with its URL filtering and whitelisting capabilities disabled are Microsoft Security Essentials/Windows Defender.    And they try to call this a fair comparison of anti-malware products?

At least we now know the true protection level of using the Microsoft Protection Suite.  Combine SmartScreen and MSE (or Windows Defender) and you get about 100% protection.  They should just test and report that instead of artificially making the Microsoft offering look bad.

Posted in Computer and Internet, Microsoft, Security, Windows | Tagged , , , , | 3 Comments