Tuesday, February 1

Results Cloudy, Ask Again Later ‡

MSFT search has been launched. Becuase of the way I use search engines (mostly for research), it will be some time before I have my own evaluation of MSFT search. Two bloggers did some searches of their own to evaluate the new service. Their experiences are starkly different and highlight, I think, the difficulty of evaluating any search engine. It depends as much on the results you want as the results you get. First up Kim Krause of Cre8PC:

For the record, I'm freezing my butt off in my office because my wood stove is taking forever to heat up.

Following up on the news of MSN's now official, finally legitimate, no more BETA folks, re-launch of its search engine, I have to admit, I was shocked at my very first pass.

The true test is my UsabilityEffect.com site, which came out in May, and though Google indexed it with zesto, rank is a sad tale I scream about behind closed doors. No matter what rabbits I've pulled from my hat, Google has given me a complex about that site. Meanwhile, Cre8pc.com is treated like a Queen; but of course, it's been around since 1996 and well deserves Google's nod. And (ahem) Google loves this blog.

MSN Search, meanwhile, has me gasping. Typing in web+site+usability+reviews, my UE site is on page one of SERPs. Not only that, MSN scored more points for featuring my services page first, followed up with the home page. How freaking wonderful is this?

So let today, February 1, 2005, be a day you remember where you were. Look for your web site(s). Revel in the love. Sing in the rain. Run around stark naked, screaming with joy. For one historical day, let me pathetically admit, I'm paryting with MSN. Come on. WoodStock, as you know, only happened once too.

Overall very positive. Next Fred Wilson of A VC:

Last November, I did a long blog post about my side by side comparison of Google, Yahoo!, and Microsoft search. Microsoft had just launched their search in beta at the time. My conclusion was that Google was still the best for plain vanilla search because of the format of the results page, the speed of the search, and the relevance of the results.

Well Microsoft is out of beta now as evidenced by the availability of the service at http://search.msn.com. The word beta is gone from the URL.

So, I went back and redid the comparison tests. Did anything change during the beta? Yes, Microsoft got worse and the others didn't change.

Why did Microsoft get worse? Mostly because the relevance of the results got worse. They did something to their algorithm during the beta period that caused the results on the searches I did to change.

First, I always Google myself. In all three engines last fall, this blog was at the top or in the first three or results that are "above the fold". It's still that way in Google and Yahoo!, but somehow this blog is now buried half way down the second page in Microsoft's resutls.

That's enough for me but in an attempt to be fair and balanced, I tried a few more keywords. So I tried "bit torrent + wilco" on all three. The first result for Microsoft and Yahoo! was the front page of bt.etree.org which is a big bit torrent site. Google had that second. But Google and Yahoo! all had various bit torrent sites above the fold, whereas Microsoft had a bunch of other stuff that was less relevant.

I tried a few more searches and in every case, Microsoft's results page was less relevant to me than Yahoo!'s and Google's.

As I said in November, Microsoft has made a nice effort to develop a competitive product, but it isn't going to change my behavior yet.

Two people using their own search results as a barometer and coming away with polar experiences. For my purposes, I think Fred's experience would more closely match my own, but I'm not entirely sure. Kim is an expert on SEO while Fred is a VC. Kim has been trying to figure out how to push a new site that she launched to the top of search engine results using various 'rabbits.' Google has resisted those tricks while MSFT ate them up with a spoon. One could therefore conclude that Google is better because it resists gaming more effectively. Fred's "bit torrent + wilco" search is interesting. What I noticed was that on MSFT the results were heavily populated by blog posts while Google had some blog and some other information. Certainly if I'm a novice bit torrent user (and I am) I think Google's results are better for me. On the other hand, if I'm searching for what's happening in the bit torrent world, MSFT's results would be superior as the gaggle of posts track the latest bit torrent happenings.

Again, depending on what results I wanted, I would have to use a different engine. Ranking one engine as "better" than another is obviously a holistic and personal endeavor. At the moment my default will still be Google, but if I'm looking for the latest info on something I might give MSFT a try.

tags: microsoft, google, seo