Search: Less Useful Due to Massive Info Growth, the Flow?

In a forward-looking presentation at the Defrag Conference this morning, Stowe Boyd pushed attendees to think about how the Web would look by the year 2019, with the aid of seeing the massive amounts of change that has taken place over the previous decade. One of Boyd's most-aggressive comments stated that the world of search is falling apart, as the problems it initially aimed to solve have been eroded thanks to the information explosion and the corresponding ease of access to social connections in a world of real time. Without saying that social networks would render the established search giants, irrelevant, he suggested, as he has on his blog frequently in the last few years, that the "flow" will replace the world of Web pages - and change the game on search entirely.

Boyd essentially argued that social tools are in the process of changing the culture. He said people were incentivized to discover breaking news from social friends through networks like Twitter and Facebook, which makes the new "real-time Web" interesting. He further suggested that how one interprets this news to define "meaning" is what will replace search.

One of the biggest reasons he thinks meaning will replace search is that the initial argument for search engines was trying to find the few documents on the Web that were relevant to your query, and now, practically any search can deliver millions of results.

"Search is starting to fail because scarcity has been replaced by infinity," Boyd said. "We are heading toward a world where all the critical information is available publicly, and breaking news is a few seconds away - at the most. We will switch to instead relying on finding things through our social connections - engines of meaning, and the source of what is important."

Assuming social elements are going to trump algorithms and crawlers that power today's engines, Boyd said he believes that the most important dimension is now time, not space - and that for the most part, this dimension is shared.

"We are not sharing space online, we are sharing time," he said. "Our time is increasingly not our own. A shared thread of time will be the norm, and how we will get work done."

This new shared thread of time, or "flow", as Stowe referred to it, is poised to become the replacement for today's static Web pages, a new element in today's social Web, which he pontificated could be "the most defining moment of our civilization."

Video: Leveraging Social Networks to Build Web Traffic

Courtesy of YourBusinessChannel, filmed while in the UK with Ecademy, some of my comments on how being active in social networking can aid business and Web sites' search engine visibility. (Apologies for looking and sounding tired. I was.)

Twitter Gives Bing Access to the Firehose, Promises More to Come

As previewed in a scoop by All Things Digital's Kara Swisher, Twitter has enabled Microsoft's Bing search engine to have access to the full firehose of all public tweets, adding these real-time elements to the company's data pool. In a post confirming the partnership, Twitter called the onslaught of updates an "overwhelming deluge", hoping that Bing could help you find those that make sense for your search query "right now".

Solving search and discovery for Twitter Search has been extremely challenging for the San Francisco-based startup, and the company's incomplete database has led to a swarm of competition, notably that of Searchtastic most recently, who gave top billing to the fact their index dived deeper than Twitter.

This obviously is no free transaction, so it is safe to say Twitter clearly has revenue today. And more will come as the company promises the development of meaningful relationships with companies that share their vision of creating value for users - be they big companies or small ones. More on the announcement can be seen on the Bing blog.

Update: (I just received this via e-mail from a Bing PR rep)

Hi Louis,

This morning at the Web 2.0 Summit in San Francisco, Qi Lu, President of Microsoft’s Online Services Division is announcing a new beta feature that enables people to easily search Twitter’s real-time information feed directly in Bing. This new feature helps people make better decisions and more fully understand Twitter conversations by collecting, analyzing and uniquely presenting real-time Twitter content.

More specifically, the new Twitter developments in Bing include:

A real-time index of the Tweets that match your search queries in results. This feature makes it easier to follow what’s going on by reducing the amount of duplicates, spam, and adult content.

Giving you the option to rank tweets either by most recent or by “best match,” where we consider a Tweeter’s popularity, interestingness of the tweet, and other indicators of quality and trustworthiness.

Providing the top links shared on Twitter around your specific search query by showcasing a few of the most relevant tweets. Additionally, Bing automatically expands those small URLs (like bit.ly) to enable you to understand what people are tweeting about. Instead of showing standard search result captions, we select 2 top tweets to give users a glimpse of the sentiment around the shared link.

You can try out the new Bing Twitter search beta here momentarily or learn more about it at the Bing blog. Please note that this is a U.S. only feature at this time.

Facebook Partnership

As part of his on-stage discussion at the summit, Dr. Lu is also announcing a global partnership with Facebook that will bring public Facebook status updates to Bing search results. The experience will be available at a later date.

Technorati Roars Back To Life After Self-Imposed Slumber

There are a select few Web 2.0 companies who have suffered such a roller coaster of peaks and valleys the way Technorati has. Once a clear industry leader for blog search, statistics, and individual site "authority", Technorati's influence withered away thanks to an aggressive push by Google into the blog search arena, statistical gaming by number-crazed bloggers, management changes, odd product launches, and inconsistent uptime. But with a major relaunch tonight, the company has tried to throw off the shackles of the old and rise again, armed with more cash in the bank, a talented editorial staff, and a new look. All of a sudden, the site looks relevant again.

Back in early 2007, Technorati was among one of the favorite topics on this blog. You could see the tumult at the company, as then-CEO David Sifry wrote on a Tuesday in a comment here that he was "very very happy at Technorati", only to announce he was looking for a new CEO that Friday, three days later. You could see debate that summer over people trying to game the then much-watched "Authority", which counted up external links to your site in a six month period. But by early 2008, we were using phrases to discuss the company that included "totally toast", and the new Twitter generation, less than two years removed from Technorati's heyday, scarcely remembers the once respected innovator.


Rising and Falling Blogs On Technorati
But as of tonight, they are back in the game. They ditched the old metrics for attributing authority, as it was considered too static, and now will aim to reward authors for posting frequency, context and linking behavior. Interestingly, they have also introduced authority by topics, meaning that technology blogs can be compared to others in their sector, as can sports blogs, music blogs, and so on. This means that aside from the overall Top 100, sites like TechCrunch don't have to measured head to head against Huffington Post, and we smaller blogs can get a better idea of who our peers are. (See:
A Totally New Technorati.com & Technorati Media Rising)


It's Nice to Be Considered a Top 100 Tech Blog
This new ranking system is looking to be more dynamic - changing along with the real-time nature of the Web. Blogs will rise and fall, and be noted on the site. Blogs and individual posts will be featured, and "hot blogosphere items" of all topics can make the front page.


An Individual Blog's Technorati Profile
Occasional louisgray.com contributor and friend to the site, Eric Berlin, is the blogging channel editor, so we wish him well and look forward to hearing more about that role. Additionally, JS-Kit's Echo will enable comments to be placed underneath all blog listings and tags, possibly adding conversation to the data.

Technorati may not be the big giant we once thought they would be, and they will need to have some consistent successes to become a blogosphere darling again, but they are back in the conversation and worth watching.

See Also:
VentureBeat: Big changes coming at Technorati — the CEO’s perspective
TechCrunch: The New Technorati

Searchtastic Aims To Extend Twitter Search Results With New Engine

Practically everybody is putting their data into Twitter these days. With an increasing velocity of tweets, it has become an escalating challenge to find data from this real-time information archive, made even more difficult by the limitations of the company's default search engine, which only displays information in the last week to two weeks. With others, including Twazzup, Collecta and OneRiot, trying their hand at finding the information in Tweetspace, a new engine has launched today, called Searchtastic. As it's so new, it's more "search" than "-tastic", but they promise future features in the coming weeks that will help the engine improve.

Searchtastic bills itself as "smart Twitter search". On launch day, there are really two major features that would have you using Searchtastic over the standard Twitter search.

The first benefit is that Searchtastic is archiving tweets longer than Twitter search (as discussed previously, there are issues). This means that tweets from more than two weeks ago may be available on their engine, but not the standard interface, giving your more results and data to work with.

The second benefit is that if you want to hone your search by adding or removing keywords, to do so is very simple. Click on keywords in search results to add them to your next query, or click on a word you just searched to remove it.


Searchtastic Finds Tweets on "Tweetie" from @scobleizer
Searchtastic also says its ability to provide "instant access to the most followed users" on their top 100 page is a feature, but I don't care so much for that, and there are other places to find beneficiaries of the controversial Suggested User List (SUL).


A Familiar Sight on Searchtastic
I tested Searchtastic, comparing it to the standard Twitter search, and it definitely provides more total results, going back longer, to the beginning of September, which is likely when they started indexing. But searches took longer to execute, and you certainly don't get any of the "trending topics" that bubble to the top of Twitter elsewhere. In fact, searching on "LSU", a currently trending topic on Twitter, without specifying a user name, simply provided me a bunch of tweets from PerezHilton complaining that "#TmobileStillSucks". So that was not what I had wanted.


Searchtastic Finds Tweets on "Spam" from @mattcutts

Adding the term "Twitter" to "Spam" from @mattcutts
The utility of Searchtastic looks to be when you want to search a specific user for a specific tweet, or see what that person has said about a particular topic. It's not a substitute for searching all of Twitter globally, and the highlighting of "top users" seems redundant. But if Twitter Search is drawing a blank where you know there should be results, maybe Searchtastic has it.

Wanted: A Magical TiVo With Sub-Program Alerts

If you own a TiVo, you can't fathom using a television without one. The ability to have your DVR record multiple channels at once so you never miss a program, pause live programming, skip commercials, and have it constantly searching for new shows to recommend or show you when your favorite actors or directors are starring in a program, is without equal. But as Web search queries and alert capabilities become ever stronger, I've been thinking of the potential for a future TiVo product to pinpoint programming down to levels much more granular than an individual episode.

Let's start with the required elements that are already in place.

1. We already know that TiVo can pop up an alert for you when you are watching a channel if something else you have told it to record is going to possibly preempt your programming.
   
2. We know that there are many many other TiVo users who are watching TV, so there is a high probability that somebody out there on the service is watching one of the stations at any given time, and if not, TiVo has the capability to tap into that channel's feed.
   
3. We know that automated close captioning and transcription tools are gaining incredible improvements over time, practically matching natural speech.
   
4. We know services like TV Eyes are constantly searching for content within TV shows.

On the Web, I have the ability to get alerted in near real-time if the brand I am watching (or my personal name) is mentioned online, be it in a blog comment (via BackType), in a blog post itself (via Blog Search), or on a social network like Twitter (via TweetBeep) or FriendFeed. Weather programs can alert me if there is an advisory in my area, or if a temperature reaches a certain level. Stock trading sites, like eTrade, can send me an alert if a watched stock reaches a pricepoint, and if lucky, I can even have it trade on my behalf if I have sufficient funds and an order in place. Each of these is an example of what I think could be the next generation of programming discovery and recording for TiVo or other DVR services - especially if users could utilize something like natural language to specify what they are looking for. Think about the possibilities that could be obtained with an intelligent next-gen recorder.

1. Post a visible alert when the channel I was previously watching live returns from commercial.
   
   This way, if a program I was watching cut to commercial, and I switched to another channel, I would know to return back to my original show.
   
   
2. Post a visible alert for specified occasions in a sporting event.
   
   I could program my TiVo to alert me when my favorite baseball team returns to bat, or if my favorite football team had recovered possession of the ball. Similarly, I could set an alert to highlight me when a specific batter came to the plate, or if the game's score differential had decreased to a specified amount. (Example Alerts: The 49ers are within 7 of the Bills, trailing 21-14 or Albert Pujols is batting.)
   
   
3. Post an alert when a specific stock is being mentioned on a business channel.
   
   If I hold Google or Apple stock in my portfolio, I could set an alert with TiVo that would pop up if Jim Cramer mentioned the companies on CNBC.
   
   
4. The TiVo could record subsegments of a show, not the entire program.
   
   What if I could tell the TiVo to record Conan O'Brien's monologue and skits, but to skip over the interviews? That's mostly what I do anyway, by hand. Similarly, can we record Jon Stewart and Stephen Colbert, but never have to see the guests?
   
   
5. The TiVo could learn my "preferred" news network and watch for coverage of topics.
   
   If I told TiVo I preferred MSNBC, CNN or Fox News, and asked the unit to watch for mentions of individuals, like the Clintons, or topics, like the War in Afghanistan, TiVo would watch the live broadcasts for me, and pick up the relevant clips for recording.
   
   
6. TiVo could integrate with network news' or ESPN's Special Report interruptions.
   
   While the concept of "Breaking News" has been watered down a great deal with the advent and growth of 24 hour cable news, the networks still can get your attention through Special Reports that interrupt regular viewing. But if I am on a cable channel watching something else, or am watching a recorded show on TiVo, I would want an alert that NBC had broken away to cover a critical live event.

Over the years, TiVo has become much more than just a recording vault for my favorite shows. The service now lets you search by show's descriptions to say, record every show with the phrase "Silicon Valley" in it, or every movie starring Johnny Depp. But I want more. The Web has spoiled me in terms of delivering real-time alerts and getting my attention right away. In contrast, TiVo's old-school episode by episode, show by show, hour by hour chunkitude almost seems wheezy. Even if we can't get to all my requests right away, I can't help but wonder how soon it will be before we can start getting alerts and recordings that are in smaller selections than the rules we have today.

If you could tell your TiVo to crawl shows you watch, or even those you don't, how would you customize this magical TiVo of the future?

It's Twitter's World: The Second, Parallel, Internet

It's Twitter's world. We all just live in it. I just looked out the window from my home north toward San Francisco and saw the sky tinged with a teal blue that spanned the horizon. While at this time last year, you could only see the faintest blue with a telescope, over the last several months, the blue light has grown ever brighter - almost blinding, to the point I can't even see Moffett Field from my condo in Sunnyvale. I've heard that SFO has even had to divert flights from landing on its North-facing runway after sundown thanks to this radioactive-like glow emanating from Twitter headquarters, as it has proven distracting to pilots who more than once have fallen into a trance-like stare, unblinkingly gasping in 140 characters or less. And I fear even with these precautions, it's only a matter of time before something bad happens. You can retweet that.

Try as I might to not be in this Silicon Valley bubble, I can't practically go anywhere or talk to anyone without hearing the word Twitter. "I've got to Tweet that!", one person exclaims. "Overheard!" cries another. "Hold on - let me check into Foursquare," says another guest at dinner, with the white artificial light glowing from his iPhone to his chest, tucked away behind the dinner menu. A phone call comes... "Did you see my tweet? Isn't that cool?" Even my iPhone apps, which I invited into my home, want me to tweet my high scores.

Sigh.

And if that weren't bad enough, many people practically expect that I have reserved Twitter user names for my 15 month-old twins. They can't yet talk, and their excitement around a laptop is practically limited to smashing the keys and being fascinated by the green light on the Caps Lock, but I am supposed to have secured their Tweeting future. Negatory on that geek rite of passage.

To be honest, it's not as if I am a hundred percent opposed to this change. If my life is to be "live tweeted" for the rest of my days, and future marital contracts will include a clause on whether one's twitter handles will change on the day of matrimony, so be it. I can accept the fact that this product, bluer than Viagra, which makes you type out messages as if you were stoned on marijuana at the speed of a crack addict, is habit forming. It's infrastructure - the new e-mail. It's the social networking glue that connects my phone and my RSS to various downstream networks. It is home to a community of people who have made their own "tweetspeak", who engage in "tweetups" and read books made "for dummies" to use Twitter - since clearly, it's so hard.

Regardless of whether you think Twitter is worth a billion dollars or a bazillion dollars, or think it's just Monopoly money, I can't help but think Twitter is running an end-around play on Google. While Google has the broadest horizontal index of pages, and is busy scouring an amazing mountain of sites to get their every word, Twitter, in a vertical approach, is getting the world's updates, and a similarly overwhelming number of links being shared. While I don't have the numbers behind me now, it would not surprise me if at some point soon, the absolute number of links (including duplicate links) shared via Twitter exceeds the net new URLs discovered by Google each day. And you can forget about linking instead of retweeting. That's the old way for Web 1.0 dinosaurs.

Twitter is practically becoming a parallel Internet. It may live on HTTP, but don't let that fool you. Over time, most folks may tell you the T's in HTTP stand for "Tweet Tweet".

In February, I said Facebook's success made it the social media prism, through which other activity would be referenced and measured. But the sheer volume of updates from Twitter is drowning all the downstream networks. If you have an aggregation service, be it the Facebook news feed, FriendFeed (now part of Facebook) or Ecademy's NetNews, you almost have to seek out posts that are NOT Twitter to find the diamonds in the rough.

Twitter is a firehose, not just for those who have glued their eyelids open, working hard not to miss a single update in TweetDeck or Seesmic, Tweetie, Brizzly, Hootsuite or a million other apps, but in terms of its sheer mass being able to render competing networks irrelevant. Facebook may be bigger today by a large margin, but it is Twitter that has the buzz. It is Twitter that is forcing the change in language, and making our LOLspeak the new standard.

Twitter was supposed to be simple. Update me what you are doing. But the community saw beyond that, helping make this product, best defined by its limitations, something else entirely. Products like TwitPic were made which helped users easily share images and videos. Users found ways to use hashtags to rally around events and causes, introduced retweeting to forward interesting items, and @replies to talk directly with one another. Directories like WeFollow were set up, without Twitter's help, to more easily discover like-minded users, and sites like Tweetmeme debuted to show the most popular links out there.

Like it or not, Twitter has become the standard for short communication. For the many of us who liked our own favorite service which we believed did more and had more flexibility or options, we were vastly outnumbered by the masses who are drinking the bright blue Kool-aid. Even as I, and others, may drag our feet reluctantly, we know we don't have a choice. Businesses who scoff at the usefulness of "The Big T" now recognize it is just another marketplace, and everybody is selling something - be it a real product, or a personal brand. Spammers love it. And that's a sure sign that you're on to something big.

To question if Twitter is going to crash and burn now under its newfound publicity and ridiculous expectations is like saying Bing's decision engine is going to make Google go bankrupt. You might as well rail against e-mail and hope it dies a quick death. Twitter owns you. Just check their recently updated terms of service. It's in there. And I promise you that you are a nobody until you've been selected to join the Suggested Users List. That's where all the cool people are. It's Twitter's world.

My Top Ten Favorite Google Products

As Google has grown as a company, its reach has extended well beyond its initial foundation as a massive search index. The company now represents many things - including a mobile handset platform, a Web browser, Web-based e-mail, a social network, and a wide variety of software programs. Like Microsoft in the 1990s, it is often hard to see a viable business where the company does not play a role - and a significant one at that.

With Danny Sullivan revealing Google CEO Eric Shmidt's favorite product is the Chrome browser, I began thinking about my own preferences, and thought I would share - inviting you to do the same.

1. Google Reader

Google Reader is my starting point for finding the day's news quickly. The RSS reader is the very best way that I know of to get all the blogs and news sources I read in one place, and it provides me with simple keyboard shortcuts to read through them rapidly, choosing to share them on my link blog to downstream social networks, including FriendFeed, Twitter, Facebook and Socialmedian.

As Google Reader has expanded its social capabilities, I have also recently enjoyed a near-explosion in active conversations on my shared feeds, and find I am spending even more time inside this product than in months past.

Though it may sound crazy, I believe the quality lead Google Reader has over its competition exceeds even that of Google Search's quality lead over its relative competition. I would rather have Reader and be forced to use Yahoo!/Bing than use Google Search and use some other RSS reader.

2. Blogger

The Blogger platform, now 10, doesn't get enough respect. The simple blog publishing and hosting product makes it easy for me to add new posts, categorize them, and update my templates, multiple times a day. Having moved well beyond its initial reputation of being something like a spam blogs haven, Google has put real effort into clamping down on bad behavior. Meanwhile, outages that used to impact the service have practically been eliminated.

Blogger is the platform of choice not just for my blog, but for my wife as well, giving us one place to log in to update either site.

3. FeedBurner

While the product hasn't seen a ton of updates since its acquisition a few years back, FeedBurner hasn't received much challenge (with the exception of FeedBlitz) when it comes to distributing RSS feeds from millions of blogs, mine included. On top of making sure that my posts get distributed, FeedBurner also keeps tabs on statistics in terms of total subscribers, click throughs and site visits, and enables the ability to customize each blog post with feedflares, adding additional interactivity.

4. Google Search and Google Blog Search

Google Search just does its job, period. Even as the Web has grown dramatically, Google's ability to return the "one right answer" solution when guessing what I am looking for is unmatched. It may lack the real-time capability of other sites, but imagining an alternative Web without Google search is daunting.

Similarly, Google Blog Search has largely replaced Technorati for most and is the default engine for finding new content on blogs around the Web.

5. GMail

While I have been using .Mac e-mail since well before GMail ever launched, the product changed the game in terms of what online e-mail represented. GMail, at its debut, offered storage space 20 times higher than the competition, integrated search and other features, such as labels and automatic filtering that make it both light and flexible. While other free e-mail products have gained a poor reputation online, seeing a GMail address doesn't make me turn away in scorn. I recommend that any business starting an online media strategy obtain a GMail account to centralize related e-mail.

6. YouTube

Though, like FeedBurner, not born at Google, YouTube is one of the most recognizable brands on the Web. Like Google Search, it has become the default service on the Web for what it does - enabling people to share videos and view videos, from silly family pictures to professionally designed music videos or corporate interviews. It is through YouTube where my wife and I share home recordings of our twins, and embed them on our sites. The ease at which we can port YouTube content to Facebook, FriendFeed and blogs is a big reason we use them above any other competitor.

7. Google Maps

Ever since I acquired a GPS unit for my car, my reliance on Google Maps has plummeted. But if in a pinch, if in another car, or needing to look up a route quickly on my iPhone, there is no substitute. While I once used Mapquest to find my frequently-lost self around town, Google Maps is now the trusted standard. As TechCrunch recently noted, only Google was sharp enough to recognize the recent closure of the Bay Bridge in San Francisco, alerting potential travelers accurately.

8. Google Chrome

A decade following the peak of the initial browser wars, between Netscape Navigator and Internet Explorer, we have an interesting tussle for browser market share once again, this time involving Microsoft, Google, Mozilla and Apple. (With Opera still not dead yet) The debut of Chrome, first for Windows and Linux PCs, with stable Chromium builds for Mac here as well, introduced more stable browsing, simplified favorite pages, and speedy load times. Let not the low ranking fool you - compared to Schmidt's #1 position. For me, it's a good product, but not the market leader in the way its brethren Google Reader, FeedBurner and Search are.

9. Google Desktop

Google Desktop brings the power of Google Search to your desktop files - helping to find everything from text files and e-mails to rich media content embedded in office documents. While in years past, much of its functionality could be found in Apple's Spotlight, or the Mac's integrated search in Finder, the latter is just too slow and unreliable, with Google Desktop gives you the familiar and trusted approach you know from the Web. Its ability to crawl through previous dates to see when documents were created is especially useful.

10. Google Analytics

Few self-respecting bloggers go too far away from their Web traffic statistics, and many have two, three or more packages going simultaneously, to ensure they have enough datapoints to consider themselves experts. For no cost, Google Analytics provides detailed stastics, not just for the last 4,000 visitors (as Sitemeter does), but for all visitors, letting you compare time periods, dive deep into demographics of visitors, and see trends in your publishing and content.

Close but not included: AppSpot, iGoogle, Google AdWords, Google Earth, Google Docs, Google Finance, Google Groups, Google News

What are your top ten Google Applications? Did I miss your favorite?

Twazzup Live: Best Twitter Profiles Anywhere, Powered by Real-time

Twazzup, which initially launched as an alternative search engine for finding popular items on Twitter, has expanded its reach - first with the product offering customized versions of its page for specific events, charities, or even political movements, like the upheaval in Iran, and now, perhaps in stealth mode, the company is working on a channel called "Live", at http://live.twazzup.com/, which presents the most thorough profiles available for Twitter - a remarkable step ahead of the standard offering.

Twitter profiles are pretty vanilla. Everybody's Twitter profile (like mine here) shows your most recent tweets, a quick Web link and bio, and your following vs. follower counts. But looking at this page doesn't give much indication of who is getting engagement, who the person is influenced by, or outside of the last 20 tweets, what they tend to discuss.


My Live.Twazzup.com Profile
The new live.twazzup.com service, in contrast, focuses less on the numerics of one's profile, but what actually happens when they hit the send button. As you can see on my live.twazzup.com page, the service shows top keywords and hashtags I use, my most recent tweet, who influences me, where my last few links have headed, and most interestingly, a real-time results stream of people who have mentioned me on Twitter.


A Live.Twazzup.com Profile for @QueenOfSpain
The result is a very clean and informative way to see who is participating in the conversation with any individual, and what they are promoting in their link economy.

With the rise of spam bots and autofollow or auto-unfollow services, the total numbers are getting less important over time, in my opinion. Twazzup is looking beyond the numbers and showing us the bigger picture - delivering what I believe to be a more accurate profile of who we are on Twitter.

You can find your own Live Twazzup profile at http://live.twazzup.com/. Just add your user name.

Twitterfall Launches Twitter Reply Search Engine

As Twitter develops, the service's users are pushing the envelope beyond the company's initial expectations - taking what was supposed to be vanilla status updates to a small group of friends, and extending it to include features that are practically standards at this point - including replies from one account to another, and retweets. But some of the discovery to find related items from one tweet to another can be very hard. Individual users can see mentions, and with some work, Twitter Search can be useful, but for the most part, individual updates are disjointed pieces of conversation.

Twitterfall, a project undertaken by a pair of 19 year old students in the UK, already seen as a potential news discovery engine, has now extended their prowess to include capturing all data around replies - to one individual, or any specific tweet.


Replies to Facebook Developer Joe Hewitt on Twitterfall
The new Twitterfall Reply Search gives you two ways to find replies.

The first is to enter the ID of the specific tweet. Each tweet has a unique identifier, which is the number at the end of each tweet's specific URL. For example, a tweet with a URL of http://twitter.com/ev/status/3538745942 has the ID of 3538745942. To find replies to that tweet, you would enter "3538745942" in the first box. (Results displayed here)


Replies to President Barack Obama on Twitterfall


Replies to Digg's Kevin Rose on Twitterfall
The second way is to search for an individual user, which will show you their most recent tweets. Click on any result and you will see any replies in the database for those tweets. Fun ones would include popular accounts, from tech leaders, to yes, celebrities, who have highest engagement on Twitter.


You Can See Replies to All of Oprah's Tweets By Clicking on Twitterfall
The results display the original tweet, and each successive reply, in the order it was delivered. Twitter Search, in contrast, may find you each individual reply by finding the user's @alias, but even then, "show conversation" is hit and miss. Assuming the Twitter API doesn't fail us, TwitterFall's new approach is an interesting research tool for concatenating responses. In parallel, the company also launched their own API to access the replies database.

See also: Twitterfall's posterous: See who's replying - right now.

No More Beta Codes: Lazyfeed Is Open for Everyone

Almost two months ago, we first introduced Lazyfeed, a real-time blog search and feeds engine, which has grown to be a big part of my information gathering process, following topics which I like, rather than people, or specific RSS feeds. The product's uniqueness is one of the more innovative services I have seen in 2009, which was a big part of why I practically have had a part-time job as a beta code broker, trying to gain people access to the site. As of today, that unpaid gig is done, for Lazyfeed has removed the beta wall, and opened the site to all who want to enter. If you didn't get access to the site, I strongly recommend you go and try it, as you will find the wave of interesting news and blog posts from around the Web addicting.


Topics for Lazy Me from Lazyfeed
As I've mentioned in my previous coverage of Lazyfeed, including a quick how-to video on the service, the site does two major things - first, finding blog posts and news articles relevant to me based on topics I have provided, and second, find related items to those entries I have added to my own social sites, including this blog and my Twitter account. Blog posts that have tags will generate new items, while hashtags on Twitter will do the same. This second portion is what Lazyfeed says gets you the lazy portion of the name, because in "Topics for Lazy Louis", I haven't even had to enter the topics into Lazyfeed - it just happens on my behalf.


The New Lazyfeed Home Page Is Open for Business

Ethan Gahng, CEO of Lazyfeed, uses his own product in a much different way than I do, leveraging the search engine for entertainment, sports, TV and other things, where as I just continue to feed my tech obsession like a bright orange RSS-powered IV drip. But that's what makes a site like Lazyfeed so powerful - it can be whatever you want it to be, and hopefully, without too much work. Stay lazy.

Real-time Search: What's Most Important Now, Not Most Accurate

This afternoon, at TechCrunch's Real-Time Crunchup event, representatives from many of the innovators in the real-time search space had a quick round table aimed at furthering the discussion, framed by a question by moderator Erick Schonfeld, who said that some on the panel may believe real-time search is defined by Twitter Search, while others believe it is "everything on the Internet, but with a freshness or recency component". And while many different companies, including the standard-bearers, like Google and Microsoft, are looking to take on this new challenge, how they are doing it differs greatly.

Danny Sullivan, author of Search Engine Land, said, "We need better definitions of what it is, so as consumers and users, we understand what we are interacting with. Through Twitter and a few other services, you have the option to publish in a few seconds. Maybe you call it social sharing search." (He also posted a summary of the players last night)

Some of the participants could be defined by how large a percentage of their data was initiated through Twitter, and how they worked with the data, including filtering.

"I would define it as what are people saying in real time about my topic," said Gerry Campbell of Collecta. "It's not what is most important, but it's what is in real time now."

This bifurcation of the "one right answer", often championed by the existing search leaders, versus what's most right "now" is helping to separate the old school search engines from this new breed. But don't think that the more-established companies are taking this lying down.

Google's Matt Cutts, who has been at the company since 2000, said "we have always talked about freshness of content." He relayed a history of his time at Google, saying they once had a "war room" of how they could refresh their search index as frequently as a month. By 2003, the company had moved from monthly updates to daily updates, and a few years later, in 2007, integrated the company's Blog Search product into its main search results. "We have rearchitected our system to be as recent as possible," Cutts said.

As updates flow in at an ever-increasing pace from all corners of the Web, search engines have the daunting task of getting accurate responses out there, while ignoring off-topic or harmful data, such as spam. And those who manage to get the formula right will have a serious leg up over those who don't filter well, making their results more noise than signal.

"Drinking from the firehose is a ticking time bomb," said Kimbal Musk of OneRiot. "Even by filtering 90 percent of what is going on with the Iran election, you're still only going to get a tiny slice, and a good portion of that is spam. If you don't filter content, you are going to get more and more spam." He later added, "If you stick to Twitter alone, you will have a spam-filled and biased data set."

With Twitter's API getting to a point where more and more companies are relying on it as their engine and data source, each is working of a common data set, and how they interact with the information will make the difference. And yes, Microsoft or Google may give you one result that is most accurate, but not for this moment, and not with any kind of impact from your friends or in terms of how that data is being interacted with in real time.

Sean Stutcher of Microsoft clearly stated this information is becoming more relevant, saying, "The sentiment around a link could be changing, and that might become very relevant to a user."

In an isolated search world, where an index is an index and the right answer is the right answer, that might not matter. But in real-time, it could matter immensely. As each of these companies works through their user interfaces, their data sets, and improves filters and social aspects, it should be very interesting to see how they separate from the pack and help define their goal.

Lazyfeed Poised to Debut Real-time Personalized Blog Search

The overwhelming majority of attention and innovation in the world of real-time search in the last year has been paid to microblogging, with Twitter and FriendFeed making most of the headlines. But a new tool, set to debut in the next two days, called Lazyfeed, is ready to unveil a service that aims the real-search firehose at full-fledged blogs, but has done so in a clean, personalized way, based on your own activity around the Web, and the topics you hand-select as interesting.

The world of RSS readers has been slowly upgrading over the last few years. Google Reader has become the de facto standard, thanks to its simplicity and the social nature of its shared link blogs. The basic elements remain true - you subscribe to a set of blogs, and those items fill your reader, making you responsible to read them one by one, or give up and "mark all as read". Lazyfeed, true to its name, tries to bring the best news to you, by topic, rather than by source, or by friends, as other social networks do. And the result is an extremely compelling way to find new stories that are relevant to your personal interests.


Hot Topics On Lazyfeed


Entering my data into Lazyfeed. Look familiar?
Setting up one's Lazyfeed begins as many different social services do these days - through the addition of your personal network information. I added my blog, my Twitter account, and my Delicious bookmarks. Lazyfeed then checks out your sources, finds tags, and starts to present relevant information. If you write about Google and GMail, like I did in the last two days, Google and GMail may be added as tags and relevant blog posts start to flow in, on the left side of the screen. I can click on the tag to read the posts, and as I do so, new posts that match my relevant tags take the top spot - in real time. The "topics" with new posts take the top spots, while the one with oldest data sinks to the bottom.

Helpfully, Lazyfeed makes it very easy to build out your list of relevant tags. If, for example, you selected Twitter as a topic, it will offer up relevant topics, such as Facebook, Socialmedia, Socialnetworking and Web 2.0. Clicking on any of these relevant topics feeds you, yes, relevant posts on those topics, and you can add any of these topics to the list in your account.


Personalized Topics Just for Me From Lazyfeed

Lazyfeed Takes a Look at Apple as a Tag
Lazyfeed's innovation makes the old-school blog directory, Technorati, look like chicken feed in comparison. And instead of being forced to read every single post from blogs that I subscribe to, Lazyfeed presents me with only topics that I am clearly interested in, and ends up being a fantastic discovery tool for new blogs that I may not have known. While I have personally relied on social networks like FriendFeed and word of mouth to find new sources for blogs, Lazyfeed looks like a great way for me to find blogs that publish often, which are targeted to my interests.


Lazyfeed Shows Social Networking and Status Search
When I met with Lazyfeed last month, I thought the company's service had incredible potential, not just for when you are logged in, but even for the casual, not logged in user. The company knows, in real time, what popular tags are filling up feed readers. It also knows the timing of posts, and could theoretically, dynamically, present to you the trending topics of the blogosphere, or make it simple to use a directory-like format to drill down and find blog posts on topics (like Yahoo! or Open Directory). With time they could get there, but they already have taken a big step forward, by introducing what are called "Hot Topics", much like Twitter Search's trending topics - which brings common conversations around mass media trends, from "Michael Jackson" to "Swine Flu", etc.

Lazyfeed is the most innovative approach to the real-time blog search world that I have seen, period. It's interesting enough to make me want to make this a very big part of my Web experience, in addition to Google Reader, FriendFeed and the rest. You can find Lazyfeed at http://www.lazyfeed.com.

Status Search: Updates from Your Social Graph on Facebook, Twitter

Status Search: http://www.statussearch.net/
We used to value a search engine by its sheer size. How many pages did Google or Yahoo! know about in their crawling of the Web, assuming that the engine with the largest number wins, offering a greater percentage of the Web's content in their database? But there is room for small too - helping you to find data from much more close-knit groups, like your connections on social networks. A new search engine called Status Search, in early beta, returns only results from friends' updates on Facebook and Twitter - the idea being, of course, that you are finding information from trusted sources, and therefore, potentially higher relevancy.

Status Search, created by Elad Meidar and Lior Levin, is not looking to be yet another real-time search engine in the ballpark of Twitter Search, OneRiot or Twazzup. Instead, it is just focused on friends' status updates. And this means that every single answer you get on their engine is from a connection you have made.

The initial product supports both Facebook and Twitter, but support for LinkedIn and MySpace connections, for starters, is also planned.


Status Search Results for "Movies" In My Networks

Status Search Results for "Sleep" In My Networks
Instead of leveraging the "real time" buzzword, Status Search hits another one - the "social graph". As is common with sites that are using social graphs at their center, the product helpfully suggests you look for things such as where to eat, what movies to see, what books to read, or even if there are parties.

Unsurprisingly, when I searched my connections, I didn't find too much on eating and drinking. I did get a few positive hits on movies that had just been watched, and I always found responses when looking for more techie items, like Google or Twitter.


Status Search Results for "Google" In My Networks
The results with Status Search are accurate, and personal. But as the site is just getting started, I found queries to be slow, and found responses from Twitter typically overwhelmed those from Facebook. It also isn't clear how often the updates are polled, or how far back their database goes, as I was surprised when some queries found no results or many fewer than I had anticipated.

I would also be remiss if I didn't mention that FriendFeed also lets me search my social graph, seeing more than just status updates, but also shares, blog posts and video, from just my friends. But Status Search isn't trying to pull in anything else but status updates, so they are laser focused. You can also set up alerts, so you get e-mailed every time a keyword is mentioned in your social graph. Alerts can be sent immediately, or daily, much like BackType, TweetBeep, and Google News Alerts.

To search what the world is saying on Twitter, Twitter Search is the right product. If you want to search what just your friends are saying over multiple social networks, Status Search could be a clearer, quieter, alternative.

FriendFeed Debuts Real-Time Search Spanning 50+ Social Sites

More than a simple aggregation tool and social network, FriendFeed has grown to be one of the deepest social databases on the Web, taking in information from more than 50 different social sites, including blogs, status updates, photos, presentations and video, and making it searchable. The service moved its core product to real-time a few months ago, and has now taken a big leap forward in also making its search results real-time, letting you see how people from around the Web are engaging and talking about topics, covering much more than "just Twitter", which so far has been the go-to destination for real-time response.

Best of all, the service isn't asking you to change the way you do searches, and all saved searches on FriendFeed work, but they now execute in real-time and continue live updating as new entries are added to the service.

For example, I could now embed a vanity search in my blog and see it in real time, thanks to FriendFeed.



As you can no doubt guess, popular discussions that have dominated Trending Topics on Twitter would also see rapid updates on Friendfeed - as FriendFeed acts as the superset for social activity. See for instance: Obama, TweetDeck or Michael Jackson.

FriendFeed's user base is still smaller than the most popular Web services, including Twitter and Facebook, but they are executing on making a feature-rich alternative. Moves like today's addition will continue to separate the innovative team from others who are still putting their full efforts into maintaining stability.

BackType Takes On TweetBeep With BackTweets Alerts

Two of the most valuable tools I have been championing over the last several months are BackType and TweetBeep. The first searches through all comments around the Web for keywords you define, while the second watches Twitter search and delivers results to your e-mail. In tandem, the pair can essentially run on auto-pilot, giving you a real-time look into what the world is saying about you, your company, or your product.

Today, BackType extended its service to let you monitor Twitter, searching for links on Twitter that contain keywords you define. Like with its core service, and with TweetBeep, the new BackTweets Alerts system will pass on mentions from within Twitter to your e-mail, as quickly as you like, from immediate, to daily or once a week.

As I told John McCrea from Plaxo when I met him for Social Web TV last week, TweetBeep has made it so I don’t have to sit in TweetDeck all day and monitor keywords. Similarly, BackType doesn’t have me chasing through comments and reading every post, but I don’t miss much because I let these strong search tools do the work. (That's also a big part of why I said there is no information overload)

Will BackTweets Alerts be so good that I can do away with TweetBeep and turn to BackType for everything? I'm going to sign up and find out.

Google’s Apps Surround Search, Pulling a Reverse Microsoft

As the discussions around Bing continue, I found myself often thinking of how the product would need to not just be marginally better than Google search for me to switch, but dramatically better - not due to an inherent bias on my part, but because of how the landscape has changed. Under our nose in the last decade, Google has grown to represent much more than just a search engine – essentially recreating the major pieces of the operating system experience around their crown jewel, with a large number of hooks that have me choosing their search over others, even if competitors are “good enough”. And the more I think about it, Google has pulled a “reverse Microsoft”, not so much in an anti-competitive sense, but in terms of how they have created customer lock-in.

Microsoft is in an unenviable position many times when it comes to the Web. Nearly two decades of underperformance on search, portals and Internet access have the Redmond giant constantly changing its approach as it tries to fend off more nimble competitors. But as we all know, it ripped its way into the Web discussion in the mid to late 1990s through leveraging its operating system monopoly to push Internet Explorer to the #1 position against Netscape, adding onto its leading position in office productivity suites, and yes, the OS.

Microsoft customers could be seen climbing the ladder of Microsoft lock-in from the bottom up – starting with the operating system, adding the office suite, the e-mail application, the Web browser, and sometimes, the MSN portal or search engine.

In contrast, Google started with its search engine and has worked the other direction – adding a formidable e-mail option in Gmail, an office suite with Google Docs, a Web browser with Chrome, a portal with iGoogle, and many utilities designed to make us come to Google as our information engine – from Google Maps and Earth to Google Reader.

Meanwhile, as Microsoft came under fire for bundling its browser as part of the operating system and forcing OEMs to preload it and not its competition, Google went out and signed deals making its engine the predetermined default in practically all non Internet Explorer browsers – including Mozilla’s Firefox and Apple’s Safari browser, making it a formidable barrier for other engines, Microsoft's included, to gain share. And as we discussed previously, late last year, in the debate on mobile phones and Web browsers, where I argued that the new tactics will be “all about the hooks”, there’s no question that Apple’s iPhone, combined with Google’s Android platform, will extend the share of Google’s engine even further on the mobile Web.

So far, Google has escaped serious drama in the world of anti-trust, a benefit its competitor from Redmond does not enjoy. As Microsoft is forced to contend with pulling its browser from the operating system in Europe, or seeing flack for Bing taking over as the default search engine in Internet Explorer 6, Google continues to make deals that make its kingpin position even more secure, and add new applications that make me even less likely to leave the site. After all, if I switched to Bing, I would still have no intent to ditch Google Reader. Microsoft has never really competed with Google Maps, making that a no brainer, and though Google’s office suite online isn’t the best or biggest, arguably, at least when I am using Microsoft’s office suite, I am doing it offline, away from the real battlefield of tomorrow.

When Google first debuted and we were measuring its success in the speed of response, or simply by the number of pages in its index, I don’t think we foresaw how it would turn one of the most aggressive tech monolith’s advantages on its head. While I recognize Google Search might not be dramatically better than Bing or even Yahoo! Search at this point, once you take the brand names away, it’s the hooks that have got me.

Spokeo Debuts Social Mining Tool Based on E-mail Addresses

Spokeo, having long since given up its initial plans to aggregate all your friends' activity alongside RSS feeds, in what could have been an interesting mashup of FriendFeed and Google Reader, has meandered ever closer to the darker side of Web, positioning itself as a tool to uncover details about people, most likely without their knowledge, and certainly without their implicit permission. The company took another step forward with this plan today, unveiling an "email research tool" that lets you search and find all the online activity associated with that address - crawling more than 40 networks.

If you remember your Spokeo history, back in October of 2008, the service declared Web 2.0 "over" and ditched its initial plans of posting trusted friends' updates with ads everywhere. So of course, this new research tool is not free.

To save you the trouble, I ponied up $35.40, the equivalent of $2.95 a month for 12 months (which automatically renews), to see what damage I could do by querying a few friends e-mail addresses. Luckily, I haven't had to pay per address, but can query as many as I like, to find what secrets they might be hiding.


Looking up one friend using Spokeo's research tool...

Results! Spokeo turns up Rob's activity around the Web.
For those friends I already follow on other aggregation services, the results were hardly eye-opening. The data was accurate for me, gathering results from every network from Friendster to Flickr, LinkedIn and a ton of stops in between. Spokeo also was able to dredge up the personal details of folks like Wayne Sutton, who was spotted on 16 different social networks.



Wayne, unsurprisingly, was found everywhere.
For friends not quite as active online, the service was not always as accurate. One friend of mine was falsely reported as a Hector Hernandez in one case, and in another case as Holly Hensley. Neither was the right profile. But for another, it found a Facebook profile I didn't know existed.


Spokeo found my blog posts, tweets and links to me.

Spokeo also found my registrations across networks.
Unlike some Web 2.0 companies who have either clung to broken or missing business models, or instead, closed their doors, Spokeo is trying to make a go of it, leveraging the technology that once looked up your friends, and now, for a fee, will look up anyone you like. They won't even know. After all - isn't it their own fault for putting that data out there?

Palm and Bing Triumph Over Low Bars They Set for Themselves

Amidst the buzz from Apple's Worldwide Developers Conference that took over the tech news world today, in the shadows, something very weird has happened. Companies that were once market leaders, and then, later, laughed at as the ugly stepchildren in tech, are being championed once again. And this time, they are being lauded not because they are the best necessarily, but because they are doing a good enough job to avoid ridicule - a good enough job for us to praise them for not completely being full of fail. Of course, I'm talking about Palm's new Pre and Microsoft's latest search entry, Bing.

I have never seen, touched or tasted a Palm Pre. I've heard they are hard to come by, and they were only available initially to a select list of reviewers. So far, the reviews are good, and the Pre is being seen as a real challenger to the iPhone. While we all ignore the traditional market leaders, like Nokia, Sony Ericsson or Motorola, it is Palm, Apple and Google who have us talking about phones. And Palm, despite being brand new and having an application store with a few dozen applications, compared with the tens of thousands on the iTunes Store, is giving people pause because it is even coming up at all. We had left them for dead, and they are rising like Lazarus, becoming part of the conversation, when most of us expected them to just go away.

Similarly, Microsoft's Bing continues to get positive writeups as people realize you can search with it and not suffer a fatal disk error. Over the weekend, a site was built that showed Bing going head to head with Google and Yahoo!, in a blind test, and doing very well, more so than likely any of us would have anticipated. While it still was losing, the results, showing it competitive at all, were enough to change our perceptions a bit - after years of seeing Microsoft unable to impress us while under assault from Mountain View.

It's quite odd, really. Maybe it has something to do with the fact that much of the tech blogosphere likes so much to rally around failures that when something miraculous like Bing being ahead of Yahoo! search in market share for a 24-hour period happens, it becomes front page news - or that the Pre actually had people waiting in line for its debut. We were all prepared to write about the disaster, so when something resembling a middling success struck, it caught us all by surprise.

But a few days of positive headlines and friendly nods cannot a market share leader make.

Palm wades into a hostile cell phone environment where Apple leads in mindshare and has the ears of thousands of developers looking to make serious coin. Google has extended their reach to many different applications beyond vanilla search - from YouTube to Google Reader, GMail, Maps, Earth, Docs and so much more, making replacing a search engine or swapping out mobile phones, once a choice has been made, that much more difficult. As I wrote on FriendFeed, and said in Jesse Stay's first podcast tonight, even if the Pre and the iPhone were feature equal, it's the integration with iTunes and all its applications that makes the difference for me now. I'm invested in this platform, and I'd venture a guess that a ton of other people are too, AT&T or not.

As for Bing? Google is the default search engine in my Safari. I trust Google to get the right results, and even catch myself searching it for results from my own blog posts' history often. Bing is a cute alternative - something to use if Google ever ticks me off, or magically, goes down for any extended period. But it hasn't delivered the "wow" experience that tells me a good reason to switch. Microsoft may have built a better mousetrap than their previous models, but they don't have enough bait.

Microsoft and Palm. One, the current and longtime leader in operating systems and office software. The other, the onetime leader in handheld operating systems. Now, today, both have tarnished brands looking for a little spitshine. They may have gotten a little buffing, but not enough to have me seeing them in a new light.

Google Gets Serious About Blog Search. Look Out, Lijit!

When I met with the Lijit team in Boulder earlier this year, they asked an important question around brand loyalty. While I was already a user of Lijit, did I feel a loyalty to their brand over others? I thought a bit, and answered honestly, that I didn't (or at least not much). I said that if Google were to deliver an equal or better product to theirs, they were already the trusted name in search, and I would at least give them a shot. Today, Google has done just that, taking an element from Blogger In Draft, and releasing it into the wild. It's one of the many steps required to complete the marathon I told you about last week.

As Blogger announced today, the new Search Box gadget searches not only your own posts, but Web pages you link from your blog and blogs in your list (like a blogroll). This is quite comparable to Lijit's offering (see right sidebar), where users can search my blog, my social activity, and my blogroll.

I haven't yet installed the Google Blog Search gadget on my own blog yet, partly due to the fact I am using an older template, and don't yet want to break what works. But you can see the widget in action on Product Manager Rick Klau's blog on the right side.


Searching Rick Klau's Blog for FTP


Searching Links from Rick Klau's Blog for FTP


Searching the Web for FTP
For example, searching his blog for "FTP" displays the results in line on the blog without a pop-up window or going to a new page. I can click "Linked From Here" to see pages that have linked to Rick that also mention the term "FTP", or lastly, I can search "The Web" at large.

It is an elegant solution, so far as I can tell.

So what does this mean for Lijit? Lijit is striving to find a way to measure influence, as well as to develop an advertising network that is trusted for bloggers that use its products, so this doesn't mean the mighty Google swoops in and erases Lijit of the face of the Web. But Lijit will have to work hard to better define itself and make it differentiated from what Google offers - especially for Blogger users who haven't yet considered moving to WordPress.