Back in 2009, Topify emerged with detailed Twitter follower notifications, which made the service useful at a time when Twitter's own notifications were text only and didn't provide any information about the user. Created by Ouriel Ohayon and Arik Fraimovich, the app made it easy to send direct messages through email replies and bumped up its feature set later in the year to include follower details in the subject of the messages after Twitter similarly went to detailed HTML in their own updates. But in the last two years, as Twitter has expanded its own feature set, the company has continued to have a rough time satisfying developers, who see changes made by the microblogging company appearing arbitrary and without warning.

Today, after a week's worth of frustrations, following a change to Twitter's back end which stripped required headers from emails available to developers, Topify said they are going to be shutting down the service, rather than chasing after Twitter's continued meandering API roadmap.


In an email sent to all registered users, saying the service would be discontinued on August 5th, Arik wrote that the company had decommissioned "X-Twitter headers", pushing developers instead to the Streaming API. In Topify's case, Arik said, the only option was to use a "Site Streams" version of the API, which is in beta and has no exit date. With clear frustration, he said he was done playing around.

"Considering this last episode and other actions by Twitter in the past year, I have no desire to expriment with their beta offerings. Not only this can result in unstable service for you, they might just shut it down one day," Arik said. "Topify was conceived as a response to long frustration with useless emails. Emails that you couldn’t process from your inbox, emails that had very frustrating mobile experience. Topify was sort of experiment, to see if it can be done better. Judging by your response and adoption, the experiment was successful."

As Twitter has grown, and the user base has shifted away from geeky early adopters, Twitter employee Phil Pennock (@syscomet) said a higher population of their users are marking incoming email as spam, even if they had requested it themselves, making email for push notifications, unreliable. He adds in a discussion on the company's developer site, "Email based trigger notifications sound great in theory and work well at small scale, but there's too many parts in the control of too many parties to debug and deal with false signal feedback. You'll have happier users with more reliable software, by not relying upon mail."

With that move, Topify's users won't get the chance to mark inbound notifications as spam because the service will be going dark a week from today - just another chalk mark on the prison wall for developers who turned to Twitter as a platform during the service's lean days a few years ago.

There's no question the market for determining personal relevance is a hot one. In addition to my6sense, which you've no doubt heard a lot about from me on this week and last, one of the more interesting players in this space is The Cadmus. After debuting in November of last year as a real-time stream filter to eliminate duplicates, the company found a solid position determining personalized aspects of Twitter, including trending topics, which launched in February, and trends by list the following month. Now, the company is branching out with an API that lets other Twitter clients tap into Cadmus for personal relevance.
   
The Cadmus At Work in TweetAgora, Tapping Their New API

If this sounds a lot like what my6sense is doing - that's because it is! my6sense introduced an attention API earlier this March at the DEMO conference, letting partners order all streams, from RSS to Twitter and Facebook, by relevance. With information overload being a serious problem for most active social networkers, and few content sources and clients doing much more than offering streams ordered by chronology, there is clearly a gap to be filled.

Cadmus is also teaming up with a lesser-known Twitter client called TweetAgora to raise the visibility of the good stuff while hiding the rest. TweetAgora highlights the opportunity as "Find the goods." and "Filter the junk.", including muting tweets by keyword, muting individuals, conversations, and blocking specific services, including Foursquare, Gowalla and Formspring.

Muting individuals has been closely associated with Twitter client Brizzly, from ThingLabs, and SocialToo, Jesse Stay's company, who I advise, also offers the option to mute or unfollow people by keyword, and to unfollow people based on the service used to update Twitter.

Cadmus' ability to launch an API with an active partner is a solid start, and the company is actively seeking out partners. You just might hear about some of their news tonight during an event with Seesmic.

To download TweetAgora, featuring Cadmus, head to the iTunes store here.

Disclosures: I am vice president of Marketing at my6sense, an assumed competitor of The Cadmus. I am also an unpaid advisor to SocialToo.

Google Buzz celebrated its 100th day of public existence at Google I/O Wednesday with the introduction of its much-awaited API, and the immediate adoption of that API from some interesting showcase partners, including Seesmic, TweetDeck and SocialWok. In the last three months of Buzz's availability, I've discussed how Google had aimed to avoid offering proprietary standards for developers to point their applications to, choosing instead to work with open tools including OAuth, AtomPub, Activity Streams, Pubsubhubbub, RSS and WebFinger. On Google I/O's first day, the company showed the first fruits of that labor, giving the fledgling network the opportunity to tap into the same developer community that other services, including Twitter, Facebook and LinkedIn, have enjoyed.

Chris Chabot (@chabotc), a developer advocate for Google, explained the role of Buzz at Google Wednesday, saying that "The Web is better when it is social," chronicling the rise of social activity to the point where having a social profile online is as common as having a mobile phone. "The Web is becoming a social place. It is moving from a static place to one about people sharing experiences, playing, working and talking together," he said.


But as Google's social team has chronicled several times, including Chris Messina's talk on Activity Streams at SXSW this Spring, many of the different destinations on the social Web speak different languages. Rather than inventing a new language, Google wanted to utilize existing standards to avoid such headaches, and make Buzz part of the Web's infrastructure, "not a special destination."

In April, after one particularly nutty report was issued on Buzz's aggregation statistics, we proved the majority of Buzz's active content originates on that network. The new Buzz API makes it easier for third party clients, such as Seesmic and TweetDeck to push comments, likes, rich media and other updates to the network, but also to consume content from Buzz in users' favorite application. This seemingly minor move - one already seen by Facebook and LinkedIn, puts Buzz on par with those historically larger networks, and ahead of scrappy startups like FriendFeed, which never gained such prominence.


Marco Kaiser (@marco) of Seesmic, who demonstrated his company's application pulling and pushing data from Buzz, said Seesmic wants "to support as many services as possible to as many products as possible", and as of today, all of their products, including the Web version, Desktop and Android clients, let you add Buzz as an account and see Buzz content alongside Twitter and other updates. This caps a busy week for Seesmic, following their Monday addition of Foursquare and other updates.

While Twitter has had an off and on relationship with its own development community, the ease of developers creating applications and Web services has led to much of the service's growth. Buzz, assuming expanded adoption of their API by other partners, could receive the same kind of boost that so far has been lacking in the service - seemingly siloed despite so many promises to connect with the outside world.


In addition to the announcements by Seesmic and other partners, Google introduced a testbed for developers to try out the API, which you can find at http://www.buzz-demos.com/. The test site lets you display your stream, add, edit and delete content, follow and unfollow others. It surely won't be all that long until many of the automation tools that have existed in Twitter's ecosystem make their way to Buzz.

As someone who long begged for services like FriendFeed to be supported by major clients like TweetDeck and Seesmic, but didn't see it happen, the rapid adoption and seeming ease of the Buzz API is a huge endorsement for the future of the network. It could have something to do with Google's name being attached, or the ease of the standards, but the potential is there nonetheless.

Google's official post on the new Buzz API is here: Introducing the Google Buzz API. You can find me on Buzz here: http://www.google.com/profiles/louisgray.

In November, when Twitter COO Dick Costolo (@dickc) promised ads that we would "love" were on their way to the popular status update service, I promised to embrace the move, assuming the content was relevant. Now that Twitter's "Sponsored Tweet" model has been revealed, and promotional updates can be seen in various search results, we're seeing the tip of what should become a consistent and growing revenue stream for the much discussed San Francisco start-up.

The news of the Sponsored Tweets platform came in a week of delivery for the company, who held its first developers conference on Wednesday and Thursday, talking to the somewhat shaken community, explaining their direction and plans to make more opportunities for them to tap into the company's massive real-time ecosystem. The week also saw them begin the roll-out of their @Anywhere platform, bringing elements of Twitter to the rest of the Web, this Web site included. (For example, mouseover @louisgray or @rsarver and see what happens.)

The most interesting news out of Chirp came from two things: the promise of a new API that delivered live content streaming (See @jesse: Twitter Announces Live Social Graph Streams) as well as the delivery of a new option for developers, dubbed annotations. (See @scobleizer: Developers: how will we all get along with Twitter’s annotation feature?) The combination of these two pieces means not that Twitter will be reducing the need for developers' products, but giving them more access to more data more quickly, with the option to extend information well beyond their famous 140 character limit.

For some, the metadata around Twitter's data will always be more interesting than the short updates themselves. We can know, for instance, when you said something, the location from which you said it, what client you were using, and whether it was in reply to someone else to continue a conversation. Those are things we have taken for granted - the basics. With annotations, we can now go well beyond. Additionally, the elimination of latency and polling should let aggressive clients like TweetDeck and Seesmic get even more robust, as they can focus on adding more features, not on band-aids to work around what have been slow elements in Twitter's infrastructure.


Despite my not having attended Chirp, I saw the onslaught of updates from those participating and attending through their various streams. Their viewpoint of the service having just completed the event looks much stronger than it did going in, when the news of official mobile clients from Twitter threatened to drive down developer morale. Hopefully this means future releases that are even more innovative which I can get to review here.

As for those ads? Right now, they are incredibly hard to find. Twitter is starting slow, letting big brands be the testbed before opening up to the world, as AdWords has. You can find a Starbucks ad when you search for coffee, for example, but for the most part, you won't see a Sponsored Tweet unless you try hard. Over time, I expect this will change, even as Twitter moves to make them more pervasive in search results and in third party clients. So not only am I fine with how they're being run so far, but their impact is small - except to show that Twitter is serious about revenue, just like we always hoped they eventually would be.

After the announcement of the ad platform, I got an e-mail from one reader, who said, "Still waiting for your post on Twitters Biz Model." When I pointed to my November post, asking "... that was in November of 2009. Do I need a new one?", he responded: "Ha! Touche. I forgot that you're one of the few who actually write presciently."

I can't say it's a level of prescience. It's just common sense. And Twitter, despite its challenges, is progressing on the right path. In a few years, you just might look back and say, "Remember when?"

Disclosures: I am an unpaid advisor to MyLikes, a company which also allows Twitter users to post sponsored Tweets in their stream. I also advise SocialToo, which would benefit from the new API options, and is @Jesse's company.

As Google Buzz users discover their own approach to using the service, and being selective as to which programs they pull into the site, many have been dissatisfied with the way Buzz has handled Twitter. Unlike other aggregation services, including Cliqset and FriendFeed, Buzz's imported tweets are not imported in real-time, and can seem very disjointed - made worse by their being imported in clumps. Today, one Googler explained they were going to solve the clumping issue in what was termed a public experiment, making things "less awful" in the interim before more capabilities are released, including filtering of imports by source - a popular option seen on sites like FriendFeed and Facebook.

Why Buzz does not pull in tweets in real time is extremely curious, given Google's much-publicized relationship with Twitter, their focus of late on recency in search, and of course, Buzz's centering around standards, such as PubSubHubbub, a big player in the real-time world. That Twitter does not support PubSubHubbub is obviously one contributor, but other services have managed to find a way to get Twitter updates with practically zero delay, through leveraging Twitter's APIs. The result is a truly unsatisfactory experience, contributing to many people removing Twitter as a service feeding Buzz.

The way Buzz polls Twitter for updates now can mean older updates get pulled in as new along with more recent items. To date all "new" updates were added concurrently, but now they will be redistributed using the timestamp of the tweet from Twitter, in effect shuffling those updates throughout the network. This will make Buzz's mysterious "unread" counts even less relevant, and should be seen as a band-aid.



Josh Wills, an engineer on the Buzz team, explained:

"Personally, I really like Twitter, and I want better integration between Twitter and Buzz. My hope is that this is a temporary change until we work out a way to play nicely together that is beneficial to everyone."

Josh went on to say that the changes were intended to "make things less awful" while things get "worked out" to make the tweets appear in real-time. Whether that's a business development decision or an engineering one, to make things "play nicely together" was not explained, but he added that he believed "real-time twitter updates don't make sense w/o the highly-anticipated source filtering."

The ability to hide sources, such as Twitter, from one's feed in Buzz will be as important as it has become to hide services in FriendFeed, or to stop seeing Farmville updates in Facebook, should you not be interested. While Buzz has been fairly close-lipped in terms of talking about future features, Josh's comment referring to the filtering shows it's on their radar, and no doubt coming, rather than being speculated about endlessly by hopeful users.

Older Tweets that do enter Buzz can of course be made more prominent if they gain conversation, and therefore, get bumped to the top of your feed - but from what I've seen, most engagement comes on native items on the network.

Content is coming at us from every direction in the form of streams, from blogs to tweets to social networks. These streams are combining to become information waterfalls of noise from practically every direction - noise because they are all shouting for our attention and demanding we choose them over updates from somewhere else. For some, this crashing sound has become information overload, as the demands from our data exceed the time we have available to give everything its appropriate attention. For the last nine months, I have been talking to you about my6sense's application for the iPhone, which watches my activity and senses what is most relevant to me, from my RSS feeds and my social streams. At the end of the last year, they became a Paladin client, and I have been working with them closely as they improve their digital intuition. Today, at the DEMO conference, they announce their new Attention API, which lets developers and service providers capture personalized relevance - and this, the ability to see my6sense-like sorting and impact throughout the Web, is what I have been waiting for. It is this Attention API that will help quiet the noise and start delivering clear signal from new places.


As you know from my prior coverage of my6sense, and likely your own experience with the application, my6sense's intention is to surface the highest quality, most relevant information, not from the crowd, but for you. It watches what you read, and what you don't read. It watches what you share, and what you don't. It watches how long you read, what articles you open, and what you skip. And over time, effortlessly, you will find the content getting better and better. Now, one of my first stops on my iPhone after time away from the desktop is to my6sense, so I can just scan a handful of headlines and know I didn't miss anything. The introduction of the Attention API from my6sense promises to take that personalization to the Web at large, should software engineers and content publishers be interested in delivering personalized streams to their users, no matter the type of content, without requiring them to fill out detailed surveys, or explicitly indicate what they like, and what they don't.


Examples of Prioritized Streams on VentureBeat and the WSJ
In the past few months, you have seen increased focus on relevance. Google Reader has made significant steps forward with their addition of "Magic", both on their desktop client and most recently on the mobile phone. Google's search engine has started to personalize results for you, including links from your social circle. Cascaad also talked about personalization of their social media application for the iPhone, and Twitter client Twazzup is trying to find "highlights" for you based on who you interact with the most, and other factors. But my6sense doesn't see attention and relevance as simply features - but instead as the core mission of their product. That other services are also trying to solve this issue speaks well to the need for publishers to provide better user experience to their customers and make their products even more appealing.

As I introduce my6sense to people, the most common requested product features are for its running in other environments than the iPhone. The company has promised new platforms are in the works, but the addition of the Attention API does more than bring my6sense benefits to new handsets, but to new environments altogether. While the company already made a big step to start filtering Twitter and Facebook streams by relevance in the application, you can imagine how content publishers, from vertical sites to industry news, new social networking clients and blogs could offer personalized streams, based on relevance, without having users jump through complicated hoops - or simple annoyances, with "more like this" buttons, thumbs up and down, likes and dislikes.


Examples of Apps That Could be Impacted by an Attention API
As my6sense promises today with their announcement, partners looking to offer tailored, personalized streams would connect my6sense's digital intuition engine to the streams they wish to personalize, and then let my6sense know how users are expected to interact with the streams. My6sense would then return personalized streams for each user, regardless of feed type. If partners sign up, the silos of unfocused, unprioritized data that we live in today should change - into a more focused Web that delivers content ranked for every visitor. With more people tapping into social services, and trying to follow more people in more places, the opportunity to cut through the noise and find the important information, taking it to the top, could be a critical competitive advantage. With today's launch, I look forward to seeing which applications and sites will step up first.

Disclosure: my6sense is a client of Paladin Advisors Group, where I am Managing Editor of New Media. My comments on the company's product are always independent, and do not pass their way in advance.

Over the last few years, I have enjoyed watching new sites spring up built around conversations and social engagement. These new social networks, be they about friends (Facebook), business colleagues (LinkedIn or Brazen Careerist), shared interests (MyLikes), shared items (Google Buzz and FriendFeed) or even shared purchases (Blippy), are primarily architected around posted items, and various groupings of gestures (likes) and comments. Practically all of the best sites are now following this model to varying degrees. But some are doing a better job than others of alerting you to your being mentioned or pulling new voices into the conversation. I believe that we are on the cusp of even better improvements to connect people across the Web, no matter where they are.

Take, for example, a basic share on Facebook. Today, when a shared item is added to Facebook, you can add a person's name to the item through their own twist on the familiar Twitter address of an "@reply". You can also tag photos or notes with people's IDs and they will be alerted, either by e-mail, or in their message box.


That's a great addition. But if I am making a comment on this entry, I can't refer to anybody in the same way (an @reply), and nor can anybody else. This is the same issue on FriendFeed, where I can copy a connected friend on a native entry, but can't do any kind of @reply to add them after the fact, nor can anybody else.


Two sites that do this very well are Google Buzz and Blippy.

Google Buzz has it set up so you can notify somebody they are being talked about by adding the @ symbol ahead of their e-mail address. For example, you can tag me by writing @[email protected] in the middle of any thread, and it will resolve to something that looks like @Louis Gray. The catch here is that you need to know the individual's e-mail address, have previously messaged them before through GMail, or you have to open up the person's Google Profile and use their username ahead of Gmail.com to get it right. It absolutely works, but is clunky, with too many steps.


Blippy's is the very best I've seen so far. Forget about e-mail addresses. If you want to reference another Blippy user, just enter @ and their ID, and a link is added to their profile. For me, as usual, that's just an @louisgray away. I will get notified you mention me, and no doubt, jump into the conversation, or at least see what you were saying. I used that just today to pull Jason Shellen of Thing Labs into a conversation on Brizzly, and in minutes, he was there.


By using this kind of @reply functionality, it takes out some of the guesswork and overdone ego-searching and vanity-searching from network to network. Just as Twitter has integrated mentions as part of their service, this mention functionality is becoming part of how we communicate - and I think we're about to see this taken up a notch thanks to two movements - one being WebFinger and the second being Twitter's @anywhere platform, even if it's not the goal right away for the latter service.

WebFinger is essentially supposed to leverage your e-mail address to aid in providing one true identity that is yours, across the Web. Google Buzz leverages WebFinger to tie back your mentions to your GMail account. If more services were to leverage WebFinger, I could tag your "true identity" in a comment, a blog post, a tweet, a photo, or practically anything else, and there would be no ambiguity in terms of whether I meant you or another person with your name (just like the easy confusion between Louis Gray, tech geek blogger and Louis Gray, Osage Indian senator). The open standard, being promoted by Google engineers, and some others, leverages public profile data to be made complete.

Meanwhile, Twitter's @Anywhere platform is looking to put a thin skin of Twitter on top of some major partner sites, at least at first, letting you follow people from 3rd party sites or let you perform Twitter-related actions outside of Twitter.com. I can see a future where this functionality could in theory be embedded as part of major Web browsers, or via plugins, or on enough sites whereby my mentioning of @ and then your Twitter ID would tie back to your Twitter account. Once that gets turned on, you can forget about just counting @replies and @mentions on Twitter.com, and you would instead get, for example, 2 @mentions from ESPN.com, 3 @mentions from The Huffington Post and 1 @mention on TechCrunch. At this point, Twitter's @anywhere could become the @mention and @reply engine for entire Web.

Any good social media maven knows the best ways to search for their own mentions and see if people are talking about them or linking to their content across the Web. I get real-time notifications if I get mentioned on BackType, and I use IceRocket to see links from around the Web or mentions on Twitter. Google News and Blog alerts still work, even if they seem stale. And yes, Technorati is still alive. But if more sites leveraged the kind of @mention and @reply functionality in a great way, as Blippy and Buzz are starting, and Twitter and WebFinger are promising, we could have an even better, more connected, more participatory Web without demanding constant searching. I'm ready.

Are you ready to see this work, @chrismessina @dewitt @ev @biz @scobleizer @parislemon @kevinrose @paultoo @btaylor @finkd @elatable @daveman692 and @pud?

If 2009 was the year of real-time Web, with practically every major service finding ways to bring content to its users instantly, 2010 is about optimizing the new real-time world, expanding interoperability between sites, finding more ways for users' content to be discovered, and taking the potential of real-time out of the status world and into the real world. Today, at the South by Southwest Interactive event in Austin, Texas, one panel asked if we were making serious progress in this vision, and if companies, feeling increased competitive pressures, are short-changing users in the process.

Marshall Kirkpatrick of ReadWriteWeb, who moderated the panel, featuring representatives from Collecta, Google, Gowalla and Microsoft, said "the real-time Web is a big, complex and multi-headed beast," adding, "almost as many people you talk to on the subject will give a different perspective."

For most, the real-time Web represents reducing latency from the time updates are published and when they are experienced practically to zero. This can be anything from updates from blogs to downstream aggregators and RSS feed readers, status updates from social networks to other points in the ecosystem, or instant alerts from the Web at large that a saved search you requested has found a positive match.

But one of the existing problems with the real-time Web that has occurred is that despite the focus by many services to solve the same problem, many have done so without delivering true data interoperability - and other services are trying to solve for real-time without having full access to users' public data.

"Back in the day, you couldn't send e-mail from AOL to Compuserve, and today, you can't send data from Google Buzz to Facebook," said Brett Slatkin of Google's App Engine team, and co-author of Pubsubhubbub. "Part of what we are trying to work on is breaking down these barriers that connect to different sites. If I am on Buzz and Marshall is on Identica and Jack is on Twitter, we should all be able to communicate."

Standards have evolved in the real-time Web space, from OAuth to PubSububbub, WebFinger and Salmon (as documented here), but that's not to say there aren't still heated debates over these standards, or even which version of standards should be supported. (See this article for a discussion of OAuth 2.0)

"I try to be a practical person, and when I hear about a family of specifications, it sounds like a family of work," said Dare Obasanjo of Microsoft. "There is clearly a place where we have a common pain that we can work on. There is a bunch of shared pain, and the way you have to get real-time service is to work on APIs, and that is a clear starting point for standards. Pubsubhubbub can help solve that problem, but I get concerned when you have to implement certain specs to solve that problem."

"These specifications we agree on should be useful on their own," answered Slatkin. "When you implement a specification like HTML, you are not buying into an ideology."

As the real-time Web's protocols are debated and deployed, so too does the application of these services. Google Buzz and Facebook have received scrutiny for their aggressiveness in converting assumed private data to public, and Netflix recently canceled an algorithm development contest thanks to concerns of assumed privacy violations.

"When talking about privacy, right now, unfortunately, the social networking market is failing, and they have little incentive to encourage user privacy," said Obasanjo. "I am waiting to see when people find what they thought were private updates as part of trending topics on Google and Bing. Users and companies are in conflict."

Obasanjo gave the example of Twitter needing its users to be public in order to drive value into the system. After all, if users were all private, there would be no trending topics, and thus it is Twitter's best interests for updates to be public. "There is a factor that if a user wants to be private, it subtracts value from the system," he said.

Beyond these concerns, known benefits of the real-time Web are scratching the surface of what could be done with more expanded to real-time data from other sources, it was argued. Slatkin forecast a time where you could query supply chains for inventory and purchase locally instead of from Amazon.com, turning economies of scale on their head. Scott Raymond of Gowalla talked about intersecting real-time Web technologies with geodata to show trending locations and the hot parties of the moment, by decaying the relevance of checkins over time. Jack Moffitt, CTO of Collecta, said a development environment for new tools and applications that leveraged zero latency was becoming "very interesting".

"All these guys are working on realizing the potential right now, working on real-time data," Kirkpatrick said. "Brett Slatkin said it was important people focus on the unforseen future that systems we worked on to support undiscovered use cases - things are going to get real crazy real soon."

Web-wide adoption of RSS and Atom standards has eliminated the problem of publishers providing their data, and tools like Pubsubhubbub are working to get data from one site to another faster. "Polling doesn't scale and you need a push notification to deliver it. It's possible we will have multiple winners, and we have to consider privacy considerations that people won't want their data available to everyone," said Moffitt.

The element of real-time is being layered across the Web, and it seems to be happening even if developers aren't completely in agreement over the tools needed to optimize the experience or if the debates on privacy versus public data are solved. And there's a lot of room for real-time to grow outside of the statusphere and to more traditional markets. The question is can developers provide solutions that don't have users running to the FCC?

When the well-respected open source advocate Chris Messina announced he was joining Google in January, many folks were concerned that his being absorbed in to the big company Borg would mean a cessation or redirection of some of his projects targeting the next generation Web, possibly in exchange of proprietary efforts to promote the company's products. Today, at the South by Southwest Interactive event in Austin, Texas, he spoke on how he and others in the community, both at Google and outside of it, are working to bring more meaning to our social networks, activity, and feeds, through extending today's data portability standards to include more information and more relevance. Messina walked through a history of the Web's publishing, from static portals of a decade ago, to today's RSS and Atom-powered sites, and suggested a future with even more information, based on streams, that tells a story.

Messina, after expressing his excitement about working with a team he believed was leading the industry in things he cares about, including the Data Liberation Front, letting data move from one site to another, said he was focused on what he called "generative structures" that were the underpinnings and DNA of how information is shared, updated and transmitted.

As he recounted, in 1999, portals ruled the Web, and people, myself included, would put their data in sites like My Netscape and My Excite, customizing these sites with headlines from third party services, primarily tapping RSS, which offered the headline of a story, a link and its description. In an era when publishers wanted to not give away their data, it was "the best we could do", he said. By 2005, a new extension of RSS was promoted, called Atom, which was still the essential concept of syndicating data from one Web site to another, but also adding an author and an identifier for the atomic bit of information.

Now in 2010, little has changed. Most news feeds of today, be they on Facebook, on customized portals, or the headline and link model that dominates Twitter, are fairly simple, and they don't indicate intent. As Messina said, "It's not all that different from the last 10 years, and that gets kind of depressing".

But what has changed is the increase of sharing rich media in these places, on platforms designed for "dead tree media". He said we should be able to show what we did, who with and why we were doing it, and that needs to happen through new richer formats for the social Web.

Activity Streams, an extension to the Atom Feed format, is looking to accomplish this by extending Atom and RSS with new aspects, including a verb and an object type. The world of FriendFeed, which supported a unified feed of 58 different services, where people could have one single stream that represented their identity online helped guide much of ActivityStreams' framework. As entries flowed to FriendFeed, they largely represented actions of posts, shares, bookmarks, reviews, from different sites. But ActivityStreams is aiming to do more than just syndicate data from one home page to another, as RSS and Atom have done for a decade, but also display intent and meaning.

"If your goal is to help people produce meaning, knowledge and culture, you have the basics for a pretty compelling social application and can motivate people to act," Messina said.

But with more streaming of information from many different sites, it can exacerbate the assumed problem of information overload - and tools need to be further developed to help us consume the data.

"We snack on information. It may feel like overload, but the tools haven't caught up," Messina said. "The solution to data overload is more metadata and we are at that point where can start generating that. We take the basic construct from 1999 and weave in some additional information - data about data."

As I outlined in my summary of DeWitt Clinton's talk on Google Buzz at the beginning of the month, Activity Streams are playing a big role in this new network, and these streams are intended to be open, not just for a company like Google, but other social networks that are tracking individual's activity and intent. The goal is to make discovery of intent data ubiquitous and transitive between sites, in the same way that RSS and Atom focused on publishing of data from one site to another.

Messina called the work on Activity Streams as iterative "baby steps", but ones that focus on getting today's rich media activity a home with a rich experience, and to make this process easy for service providers.

"If you have a Web site that has people doing things on it, and they have a feed they are taking with them, it is fairly trivial to add ActivityStreams information," he said. "Essentially we have the verbs and object types represented."

You can find out more on the continued development of ActivityStreams at http://activitystrea.ms.

As mentioned on Monday, Twitter introduced firehose access to their API for seven new partners. This evening, Chris Saad, Mashable's Ben Parr and I discussed the announcement of Twitter’s new API partnerships and implications for the company's revenue, developer and user communities.

Is this approach competitive or cooperative with the community, and how is their model likely to change over time?

Original Post Here: ET Conversations

Listen in below:

If you haven't seen a lot of applications built in the last few weeks that leverage the Google Buzz API, it's because there aren't any. In fact, Google hasn't yet rolled out any API for Buzz. According to the company, this isn't due to any backroom dealings where they plan to introduce proprietary code and hooks that tie activity to their platform, but instead, because they wanted to be sure they could first build a product that in fullness leveraged open Web standards, and start with that foundation to deliver an interoperable system that could continue to function even if Google were to "disappear off the face of the earth".

In a presentation to the Silicon Valley Google Technology Users Group last night, held at the Google campus, DeWitt Clinton, a software engineer for the company, talked to developers and other tech enthusiasts about the company's API strategy and approach to Buzz, and explained that Buzz is designed not to increase lock-in to Google, but instead, to leverage open technologies that will let data flow to and from sites without central ownership. While a Buzz API will eventually be released, it will leverage the same open standards that power it today.

"The first principle of Buzz is that we can build this on protocols that are open and free, but not centralized," DeWitt said. "Can Google disappear off the face of the earth and Buzz still works? We need to make this data federated and distributed."

On the day Buzz launched, I referenced much of the foundation for Buzz in a quick article about the open tools and APIs that "make Buzz hum". But last night, DeWitt expanded that story to include 9 major open APIs, briefly outlined below.

1. Atom

DeWitt called Atom "the lingua franca of the programmable Web today", explaining that Atom contains entries that are "well structured", and include source entry, GUIDs that enable deduplication, and specification of the content type. He said, "You are able to pass rich data in that Atom feed in a way that is more specific than other feed types."

2. AtomPub

DeWitt said AtomPub "has become the most popular paradigm for restful APIs on the Web." AtomPub expanded the original Atom format to include the ability to both create and update feeds, not just passively read.

3. ActivityStreams

ActivityStreams essentially watch users' activity and can specify rich verbs and actions within those feeds. This enables feeds for all comments posted on Buzz, all likes, or even alerts that one person following you on Buzz also follows you on another network. DeWitt's examples hint at future developments for the platform, as these specific feeds are not yet clearly visible.

4. Pubsubhubbub

Much discussed here on the blog, Pubsubhubbub reduces the need for sites to poll for updates, and powers real-time updates between services. DeWitt reiterated "the hub is decided on by the publisher" and "there is nothing Google-specific about that.", saying that the infrastructure and plumbing for Buzz has been laid for the last few years. Pubsubhubbub has been pioneered by Brad Fitzpatrick and Brett Slatkin, both Google employees.

5. MediaRSS

Developed by Flickr, MediaRSS syndicates rich media through both RSS and atom feeds, creating a structured namespace inside RSS for content and a thumbnail. Buzz leverages MediaRSS, letting you pull rich content, like Flickr photos, into the platform. Of course, PicasaWeb, a Google property, also supports MediaRSS.

6. OAuth

The product of engineers from all corners, including Twitter, OAuth was engineered "to solve a vexing problem in the industry," Dewitt said, explaining OAuth prevents the need to ask users for their name and passwords on third party sites, acting as a delegated authorization protocol that gives permission to the application. Google Buzz, like Twitter, leverages OAuth to provide authenticated access to your data.

7. WebFinger

A new-age version of the old command-line prompted, text responding Finger protocol, WebFinger aims to be a way to get public information tied to an individual, through their identity, assigned to an e-mail address. "We want people to identify themselves, and we want people to discover people," DeWitt said.

WebFinger is similar to the strategy of OpenID, but OpenID hasn't had massive adoption by end-users who have found it unwieldy. WebFinger, aiming to be less arcane, enables the independent nature of Buzz, helping to federate the data and distribute it by domain, owned by the end user. DeWitt said, "The profile lookup and notification mechanism can be in the hands of the user being addressed."

8. Salmon

Still in earliest stages of development, Salmon is an extension or replacement for the old PingBack model that had blogs informing the other about references or links. This "flawed" model only provided minimal data, and could not be verified, letting me send PingBacks anywhere I wish if I chose. Salmon's goal is to leverage what's being called "Magic Signatures", signed with a public key to prove and verify linkage.

The first approach for Salmon will be to migrate comments from aggregators to originating posts, as covered a few times on this blog. But DeWitt said that "Likes" are similar activities that could flow back with Salmon, or be used to notify users of "following" or other activity. DeWitt forecast that sites like Blogger and StatusNet would rapidly adopt and federate Salmon to transmit data updates.

9. Portable Contacts

Simply described, Portable Contacts show your information and that of the friends who you follow, providing users a secure way to get access to address books and friends lists without having to request credentials or scrape the data.

DeWitt also noted XFN, the XHTML Friend Network, and FOAF (Friend Of a Friend) as being key contributors to the Buzz technology stack today, adding that he was "glad smart people were working on this ten years ago because we are all benefiting from it now."

DeWitt, on his Buzz feed, has been talking a lot about open standards and their importance to the Google team at large. See @Jesse Stay A few points of clarification to your most recent post [1], because I believe getting the details right matters. and "The thing I find most attractive about Google Buzz is its stated commitment to open standards.", as well as his first post from February 21st, which thanked the standard developers: Standing on the shoulders of giants—a look at the people behind the protocols behind Google Buzz:

Given Google's size, there is a good amount of distrust on the Web from people who think they own too much of your data, know too much about you, or have goals that run contrary to your own ideals on privacy, communication and sharing. Not even DeWitt's detailed presentations and explanations and promises of openness and data portability will convince everyone that they are on the right path. But I personally believe the frankness and detail that is being shown here is not just promising a strong future for this individual product (Buzz), but also in extending the groundwork done for the entire Web, for products and services we haven't even seen yet.

DeWitt adds: "All of these protocols are open. They are literally also all free. They are intended to be used by everybody, with or without Google being involved. You don't have to ask us if you can use Salmon or Pubsubhububb. We have a liberal and permissive patent license."

Is Google going away? Not today, and not this year. Is Buzz perfect? No. Of course not. Can it do all the things I can do on other sites, like FriendFeed? No. Not yet. But it seems that the Buzz team has opted to make tradeoffs that favor fast shipping and openness over completeness and individual features. And if you don't trust Google, it sounds like you can do something about it.

"We are pretty adamant about not building this on proprietary technology," DeWitt said last night. "If any of you feel that it is not going in the right direction, you have the power to change its direction and Google will not stop you."

You can find me on Buzz here and can follow DeWitt Clinton on Buzz here.

Today, at the RSA conference, the Open Identity Exchange (OIX), aimed to increase trust in online identities, and backed by the OpenID and Information Card Foundations, announced its inception. In parallel, the U.S. Government is recognizing multiple technology companies as meeting federal standards for identity assurance, including Google, PayPal and Equifax, essentially securing users' ability to register and log in at federal Web sites with credentials from each of those services.

Goals of the Open Identity Exchange include building online users' trust and confidence in the exchange of identity credentials, standardizing these interactions and reducing hassle with online logins, registrations and purchases. As practically any Web user knows, frustrations with remembering scads of online user names and passwords, each corresponding with different sites with varying trust levels, can be a complete pain - no matter how much effort is taken to standardize, and the alternative, keeping one password for multiple services, which many do, has many more problems of its own.

OIX and its members are looking to reduce the problems with today's Web and move toward further highlighting open standards. Founding members of OIX, a non-profit corporation, include Booz Allen Hamilton, CA, Equifax, Google, PayPal, Verisign and Verizon.


Google's participation in the exchange follows the company's hirings of some of the more vocal advocates of OpenID and the open movement in general, including Chris Messina and Joseph Smarr. Earlier this week, a Google spokesperson wrote by e-mail that the inclusion of the company as part of OIX's launch should not come as much of a surprise.

"As you probably know, Google has long supported and contributed to the development of identity standards such as OpenID and OAuth, largely in order to increase online security by reducing the reliance on password use across websites." they wrote.

A white paper on the new OIX Web site, entitled "An Open Market Solution for Online Identity Assurance", explains how open identity technologies, including OpenID and Information Cards, serve to take closed user name and password systems deployed by most Web sites and expand them to accept identities issued by other parties, such as Google, PayPal and Equifax. Much of the paper, and OIX's mission, centers around the issues surrounding identity, including social, business, legal and emotional, such as trust.

This model of trust is explained in a second piece which defines a new "Open Identity Trust Framework (OITF)". The OITF paper shows holes in today's trust frameworks, and questions how people passing along personally identifiable information can be sure their data is protected with acceptable technical, operational and legal safeguards, while proposing a structured role for policymakers, providers, assessors, auditors, and dispute resolvers, to be sure that all participants are acting in a trusted manner. It may seem overly bureaucratic, but considering the Federal government needs to accept its findings, process is a good thing.

Lest you think this just yet another association or bureaucracy with talking heads looking to grease the skids of online growth, see the conclusion of the OITF model paper, where the authors explain a data utopia: "
Imagine 
that 
the 
OITF 
model
 takes
 off
 and
 identity 
aspects
 of 
all 
digital 
communications 
become
 reliant 
on 
this 
new
 layer 
of 
the
 Internet. 
Society 
could 
become
 dependent 
on 
this 
type 
of 
infrastructure 
for
collective 
action. 
The 
authors
 want
 to
 make
 it 
clear
 that 
trust 
frameworks
 for 
identity 
information 
portend 
to 
be
 so
 important 
for 
the
 future 
information 
society 
that
 they
 warrant 
extensive 
scrutiny, 
participation, 
and
feedback
 from
 a
 wide
 representation 
of 
stakeholders.
"

You can find out more on this new exchange at http://openidentityexchange.org. In addition, Google posted on the announcement on the company's online security blog: Federal Support for Federated Login