As I mentioned on Wednesday, I attended an update from Microsoft's Powerset team in San Francisco, to gain insight into how the Redmond machine was viewing what's considered to be a mature search market - seeing just what opportunity they believed they had in a space where Google is the king and Microsoft's offerings, whatever they are called these days, usually are viewed in a role similar to court jester, fairly or not. And while the team aimed not to drown us in a well-known ritual called "Death By PowerPoint", they offered us a background presentation to take home on USB keys, chock full of statistics. We were sworn not to distribute the slides, but permitted to use any and all data from within. Maybe it's a style thing.

After looking at the provided materials a few ways, it's clear that the Powerset team at Microsoft is looking at massive amounts of data, citing that search engine users report high levels of satisfaction with their current search provider, but waste an incredible amount of time honing search queries to find the perfect result. Amidst a digital content information explosion rich with links and an ever-growing number of Web sites, Microsoft believes that better tools are needed to help users find their answers more quickly and get on with their solution.

Their statistics, which cited Pew Internet, Comscore and Harris, among others, state:

• 49% of people perform Web searches each day
• Users are searching 23% more frequently per person, year over year from 2008 to 2009
• Satisfaction with search is at approximately 65%

So yes, search is critical, but the data speaks to a somewhat frustrated universe of users, who are spending upwards of 10 minutes on average per Web search, making anywhere from 4 to 6 search queries apiece before coming to a decision. Interestingly, Microsoft's presentation differentiates between standard searches and those that are used simply for navigation (such as finding a Web site domain), as those need only one or two queries per session to complete.

Assuming the data is correct, and Web searchers are taking multiple queries to find an acceptable action, Microsoft found that even after users come to a decision, ostensibly by clicking a search result, almost half the time the user will end up searching for the query again, with 19% performing the exact query, and an additional 30% being a partial repeat of that query, signifying they weren't satisfied with the link they had selected.

The end result, as Microsoft lays out, is that despite users saying they are happy with their search engine experience, about half say they do not expect their queries to be successful. They cited success expectations ranging from 53% to 57% in the categories of Product Purchase, Local Activity, Specific Facts and Finding Information. While more than two thirds of users say search is frequently part of their decision making process, half they time they anticipate being disappointed.

As with any Marketing-driven presentation, Microsoft tries to summarize their to do list with three buckets:

• Opportunity: Better Results
• Opportunity: Organized Experience
• Opportunity: Powerful Decision Tools

Taken in aggregate, the opportunity is as it always has been - help users find their data in a simple way that delivers a successful action, in a cleanly delivered fashion.

Searching through the deck doesn't mention Kumo, or Live Search, or Powerset or MSN, or any of the many names Microsoft has tossed around. It simply says there are issues to be solved, and suggests the opportunity is out there to create something better that helps customers and delivers a higher level of satisfaction. From the discussion on Tuesday, it seemed the Powerset guys believed they had a more relevant approach to delivering better results, and possibly in a better organized way. The struggles, of course, will be in how they can search out new customers who will find a better experience with their product against well-respected competition. Can they better optimize their results to reduce the number of times you execute a query?

I would believe that much of the blame lies not with the search engines, but the links themselves downstream. Until Microsoft, Google, Yahoo! or anybody else can be held accountable for the whole content of the Web, at the end of the day, they are passing you off to a third party, who may or may not have the answer you are looking for. It's possible to continue tweaking the algorithms to and fro, and making the search experience easier, but there will always be room for error, with the exception of fact-based searches (which may explain why Powerset started with Wikipedia as its data set).

Additionally, every search engine is working with a set of users that could be improved. While it's a faux pas to blame the user, there is a very high chance, in my opinion, that users are not really sure what it is they are looking for, and their own search methodologies could be improved. It's likely the Web has reached a stage where "good enough" is "good enough", and multiple queries or keyword tweaking will be the norm for some time.

By Mike Fruchter of MichaelFruchter.com (Twitter/FriendFeed)


Automation is key to making this work. My outpost task is for commercial use, so it's not a viable option for me to manually update all of these outposts. Some of the outposts will have to be manually updated, but the majority of them will be automated. I will be covering some of the tools and features for automation that I use in this post.

Twitter:
Image by Rejar under Flickr CC

By Jesse Stay of Stay N' Alive (Facebook/FriendFeed)

It would appear that Twitter has launched a brand new format for their new follower notification E-mails they send out to users opting to be alerted to new followers. The e-mails are now in a prettier HTML format and include the new follower's name, Twitter username, and picture. They also include the number of updates, number of people they follow, and number of followers they have. While users aren't given the option to follow straight from the e-mail, you can block the user by clicking on a link in the e-mail.

The New Look for Twitter E-mail Notifications

The Old Look for Twitter E-mail Notifications
Read more by Jesse Stay at Stay N' Alive.

On Tuesday evening, I attended a get-together with some of the minds behind Powerset, the $100 million natural language search engine acquisition slowly being integrated into Microsoft's Live Search engine. The goal? To keep the San Francisco startup natives feeling still connected to their roots, and not detached as part of the Redmond software monolith, while also giving interested tech reporters and bloggers an opportunity to see why Microsoft has any chance in the search market against its primary foe, Google. While I'll talk later about some of the statistics they offered, there was a conveyed perception that traditional reviews of search engines are flawed in their simplicity, in their context, and are not in depth enough to gain a clear knowledge of what's complex subject matter.

Discussed in detail on Senior Program Manager Mark Johnson's blog, Deliberate Ambiguity (See: "How *not* to rate a search engine"), the feeling is that reviewers commonly perform three queries before rating an engine "good" or "bad" - their own name, a hot topic of the day, and then something completely random. If the engine being tested doesn't match expectations, then it's off to the woodshed. (See: Cuil, Searchme, and other engines that have not been embraced for such examples)


Is Live Search Feeling Lucky? They Say Try It for a Week.
Instead of a simple Live Search vs. Google three query test, Mark recommended users try out Live Search for a full week, arguing that their own experiences would trump any simple demo. And as we discussed yesterday, one person's experience could be wildly different from another's, people primarily focus on the first search result, ignoring the bulk of the iceberg, and that it can be extremely difficult to infer intent and importance of results.

For example, as we discussed yesterday, assuming the working set of a search engine is Wikipedia, as Powerset's search engine prototype was, the web crawler will incorrectly determine that years associated with entries are of incredible importance, given how often they are referenced. As a result, rather than World War Two events rising to the top in importance, computers are more likely to trust results labeled '1944' or '1942' - so how do you teach computers to think like humans and correctly rate influence and importance? He adds, concluding his post, "this is rocket science."

At the quick meetup, questions around Microsoft's potential to dethrone Google were plentiful. We talked about how the once-mighty Alta Vista had once had a significant lead, and even after Google's debut, it held a corner in image search and language translation, before those too were eroded. Could Microsoft find a position that Google just doesn't do well? Could Microsoft start to be good enough such that if people tried to use Live Search for a full week, that Google would seem 'less good'? I even overheard a comment at the event that said in user groups, those surveyed preferred results that displayed a Google logo, even if the results shown were from Live Search.

The Google brand just means trusted search for many people, and even the Microsoft monolith is finding competition to be incredibly difficult. Google managed to find a market space that didn't need Microsoft, and could compete on its own revenue terms, without being frozen out by a competitor who could offer a parallel offer for free, a la Netscape. Even if Microsoft has built a better mousetrap, it is going to need to find a way to communicate that to users and deliver them a compelling reason to make a switch - and if Mark's words are to be believed, users and press aren't doing the needed research to help communicate their progress.