I’ve been watching all the uproar, and denial, from Consumer Reports (CR) dropping its recommendation of Microsoft’s Surface line, but I have to tell you I think CR might be on to something. Sure there are plenty of power users who report they’ve never had a problem with the Surface. Even when they have, which I’ll get to in a moment. They, and the press, just seem to want to ignore that CR operates off broad survey responses rather than the experiences of a select few. Of course CR’s data is backward looking, and won’t reflect improvements that Microsoft has made over the last 6-12 months. And I’ve always believed that CR’s data is biased by who they attract as subscribers. Nevertheless, it is a valid data set.
So why do I think that CR might be right about the Surface family? Well let’s go with my last three experiences. The first one was when I purchased a Surface 3 (not the pro) to use as a consumer tablet (i.e., alternative to an iPad). Subjectively I’d say that the Surface 3 did not even perform as well as my original Surface RT. It was sluggish and I thought kind of flakey. Then several months in the touch screen stopped working reliably. A few weeks later I happened to catch it in a funny light and found a hairline fracture in the screen. I didn’t remember dropping it, though I likely bumped it at some point, so this was almost certainly my fault. I looked at the cost of repair, and it was so close to replacement cost that I did replace it. With an iPad Pro. I’ve dropped that a few times, and it still is working just fine. Now take an average consumer. The Surface 3 never was a satisfying experience. It was not a physically robust device. It was not affordable to repair. How would that consumer respond to CR’s survey? Never mind an average consumer, how do you think I would have responded to CR’s survey?
Next let’s take the Surface Book. Like just about all owners of more recent Surface devices, I had the experience of my Surface Book failing to Sleep or coming out of Sleep all on its own. Even power users who have tweeted they don’t agree with CR admit they had that experience with Sleep in the past. If we talk average consumer, don’t you think that experience might lead them to be a little negative on the device and respond to CR that they’d had problems? Now my personal experience is even worse. I twice had the experience of putting my Surface Book in my backpack, heading to the airport, and having it come out of Sleep on its own. In both cases it cooked itself for hours. How hot did it get? Well, when I reached into the backpack I burned my fingers! After the second time the Surface Book’s screen was permanently damaged, with brownish yellow streaks along the right side and bottom of the screen.
By the way, I didn’t blindly keep using the Sleep feature (although an average consumer probably would) after the first incident. I switched to Hibernate for months. Then after Microsoft claimed to have fixed the problem I went back to using Sleep. And it happened the second time. Microsoft issued more fixes and now it hasn’t happened in a long time, but how do you think I would have answered the CR survey?
Lest one think that this is all backward looking, I had another experience just this past week. A friend bought his daughter a Surface Studio. A month later she said something about having lost her drawings and being unable to re-install the drawing app she was using from the store. In fact, they couldn’t install any app. Or run any store app. So he asked me to take a look. The system was completely messed up. Attempts to fix the store failed. Pretty much nothing store-related would work, and a lot of things in Windows 10 have a connection to the store. I had to advise them to use Windows 10 Reset to get the system back in a usable state. How do you think they would answer the CR survey?
I’ll contrast this with my Surface Pro 2, which has worked flawlessly from day one. Or my original Surface RT, which was fantastically reliable as well. From my narrow perspective, if you’d surveyed me on early members of the Surface family I’d be able to say they were very reliable. But I haven’t had that experience with a Surface device in the last 3 years. Does that mean I’ll avoid Surface devices in the future? No, I’m very likely to pick up a Surface Laptop. But I’m going to be brutal on Microsoft if that experience isn’t near perfect.
So before dismissing the CR downgrade consider that the Surface devices have had problems, and personal experience suggests those are not completely behind them. Microsoft, rather than being dismissive of Consumer Reports’ findings, needs to double down on the quality of Surface devices. They might also want to take another look at how they collect reliability and customer satisfaction data, because they seem to have missed that the customer experience isn’t nearly as good as their own data shows.