Editor’s Note: Part 12 in an irregular series of stories from my many years in Silicon Valley. Part 11 talked about the time I got called into HR's office to meet with lawyers over industrial espionage. This time, a story involving gray hat search engine marketing in the early days of the Web.

DMOZ is now closed.

Believe it or not, before the world of automated spiders that crawled the entire Web and ranked the results for your searches, much of the way we found content on the Internet was thanks to manual updates from an invisible army of directory editors. Yahoo! defined the initial dot-com era, with its hierarchical oracle making or breaking traffic downstream, as sites were organized and shuffled into categories by unseen text tweakers, much like the editors of Wikipedia try and keep its tens of millions of article pages up to date, with a seemingly fluid mass of edits to keep the live encyclopedia current.

But Yahoo! wasn't the only Web directory. Rich Skrenta and others, also behind Web 2.0 efforts Topix and Blekko, introduced the Open Directory Project, referred to as DMOZ, for it was hosted on the directory subdomain of the Mozilla.org site, with an objective of harnessing contributions from around the world (like Wikipedia), to build a directory, similar to Yahoo!, that could plug into any site that wanted to host one. In a time when many sites were seeking Internet traffic and riches by acting as the front page for the Web, attaching the open directory project to your portal could give you an edge and not require you to bring on a ton of staff.

As with Yahoo!'s directory, a company's inclusion in the DMOZ directory could act as a binary gate as to whether or not potential users would find you. In 1999, working as a Web marketing manager for a Web startup that offered internet faxing and conference calls, I found myself irked to see that our services were not included in DMOZ. Making things worse, the categories I would expect to see us listed in seemed slapped together and without an official owner. Given my understanding of the space and knowing our many competitors, I registered an account and requested to moderate the relevant category.

The DMOZ Internet Fax Listings from 1999 (via Archive.org)

Not too long afterward, I was given the option to update the category, including those of our competitors. As it was nearly two decades ago, I honestly don't remember if I used my company email or a Yahoo! email or equivalent, but I didn't try and disguise where I worked. My application had gotten through.

FaxCube from the year 2000.

When I did log in, I found the content in a state of abandonment. There wasn't much you could do with a site's listing. Give it a title, a link, and a short description, literally about a dozen words. It was fairly impossible to differentiate services from one another, especially in a commoditized space where the core function was pretty straight forward. But I cleaned up the descriptions for all the entries, including our competitors, to accurately display their offerings. Some offered email to fax services, while others offered the reverse. Some offered broadcast faxing. Some required a proprietary fax viewer, and others were completely Web based. That kind of thing.

When content was edited in DMOZ, edits would later propagate downstream. Sure enough, my colleagues noticed a spike in Web traffic to our main sites, with referrers coming from all the places DMOZ was set up. For no cost, I had a clear impact in our customer acquisition funnel, and maintaining the DMOZ became part of my ongoing work.

Later, DMOZ added the option to highlight two entries in the group, which were solely up to the moderator. This, of course, gave me the option to expand from a gray area, to clearly going over the line into promotion. It went without saying I thought our service was the best, and highlighted it at the top. I also chose to highlight a partner site (the Netscape Fax Center) that was white labeling our service, essentially the 1 and 1A positions.

This got even better when we soon realized DMOZ was fueling search results for AOL. The more people who searched for Internet Faxing on AOL, the better we were to see results.

DMOZ listings for Internet Telephony in early 2000. (Via Archive.org)

In early 2000, we introduced a new Web based conference call and meeting product. Addicted to the free traffic from DMOZ, my team asked me to see if I could get our content similarly included on whatever the equivalent was for Internet conferencing. I poked around, and, again, applied to be an editor for the Internet Telephony page. It wasn't a perfect fit, but it was pretty good.

Again, I got approved, and again, I added our sites, and again, I promoted our main site and a partner site to be included as the pair at the top of results. Again, we started to get plenty of Web traffic from DMOZ and its downstream partners, accounting for more than double digit percentage of our traffic to both properties. But this time, it wouldn't last long. Maybe it was the clear marketing copy promoting both services, but one day, I logged in and saw our service demoted to the standard results, with another in its place. I reversed the change, and it wasn't too long until there was a debate upstream as to whether these listings were in good faith.

PhoneCube, in all its glory, from 2000.

Not too long afterward, my moderation rights had been removed from both sections. I'd essentially been fired from my side job of gray hat SEO, long before most of us knew what that meant. And yes, my engineering peers lamented the loss of traffic, as our more organic listings, on AOL search and elsewhere, didn't carry as much weight, once we became one of the fold.

Had I just stayed content to include our services in the listings, or even just to stay a moderator of the less exciting Internet faxing space, it's possible I could have been editing text descriptions for our sites and our competitors indefinitely. But pushing our own products and our partner sites turned out too good to be true. The "Wild West" Internet traffic rush didn't last forever, and, as it turns out, neither did our products. I was laid off after we couldn't get a funding round closed at the beginning of 2001, and a few months later, the company was parceled off to become part of Oracle (see 2006 post). Some of my colleagues still work there nearly two decades later. As for DMOZ, it too shut down a few years ago, a relic in the world of Google and Facebook.

Disclosures: This whole post is a disclosure! I worked at 3Cube during this time. I work at Google now, a perceived competitor to whatever Yahoo! and AOL call themselves now. And as I work on Google Analytics, this is not a post about SEO to all you SEO/SEM folks.

With today's focus on realtime systems and 24 hour news cycles, trending topics and viral videos, our collective attention span seems to be shrinking. The news of the morning is not news of the evening. Yesterday's news is old news. Yesterday's celebrities and entertainment are forgettable and mockworthy. And today's technology might be obsolete by the time you buy it, eclipsed by a competing offering introduced as you drove home from the computer store, or while Amazon Prime shipped it to your home, while you tracked its every stop by email.

So woe be unto the companies and achievements of yesteryear, who once held lofty positions in business, made headlines with their every rumor or news leak, and broke the rules on the way to setting records. For once their time has past, proud memories rapidly decay in a world that focuses on the new shiny thing and casts them out as no longer being relevant.

It's easy to say that those who forget the past are doomed to repeat it, and yet even with tremendous tools like the Internet, resources like Wikipedia and Archive.org, onetime tech titans are rapidly shoved aside as dinosaurs, as younger people enter the job market and participate alongside more traveled veterans in Silicon Valley and beyond. Even those of us who've spent just a decade or more in cubicles and offices have seen pioneers like Sun Microsystems, Silicon Graphics (SGI), Netscape, 3Com, Palm and others fade into the electronics landfill in the sky. While just years ago, it seemed you could throw a rock from any exit on highway 101 and hit a Sun building, the once-proud company is all but gone, a speck within acquirer Oracle. Netscape's immolation by Microsoft and AOL, each in their own way, passed the torch to Mozilla and others, and nary a plaque in Mountain View marks their spot.


My own employer, Google, inhabits a number of buildings acquired in slightly used condition, be it from Adobe, or the aforementioned SGI and Sun. The main campus, in fact, used to be that of SGI, and the corporate colors seen in SGI's hardware mark the buildings of the Googleplex, with little modification. This land grab started only in 2003, at a time when Google was a mere 800 employees, and SGI was in one of their many phases of trying to keep expenses in line with revenue, seeing the abandonment of their splashy campus as one way to take down costs. Now SGI, having seen many corporate transformations since, is still around, but has little ties to its original makeup.

While 2003 is "not that long ago" to someone like me in their mid to late 30's, it's quite a long time ago for those just entering the workforce. I turned to a new colleague of mine early this year, she having just graduated from college, and asked if she didn't mind a little word association. I started with SGI. No recognition. I then asked about Sun. Again, nothing, even though we were standing in one of their old buildings. I then turned to the previous generation of search engines, starting with Lycos, moving to Excite and Alta Vista. Finally, she offered up, "Was that like Ask Jeeves?"

Only slightly alarmed, but also amused, I tried another colleague, starting the word association with Sun. "Java?" she said, getting partial credit. I asked, "Anything else?" remembering the "dot in dot com", "the network is the computer" and all the enterprise work that once made Sun one of the four horsemen of the Internet, in addition to Microsoft, Oracle and EMC, if you kept your TV tuned to CNBC in the heady bubble days. Nothing else. Further sparring with names like SGI and a bevy of Web 1.0 darlings similarly went absolutely nowhere.

Even if we look just in the past two years, you can see the fast rise and fade of companies, as the Web rapidly chooses winners and spits out their less favorite. MySpace flew too close to the sun and torched its wings like Icarus. Formspring was once a Internet doll and collapsed. Slashdot and Digg faded to shadows of themselves as Reddit and others rose. And big companies are not immune, as you can see with Kodak filing bankruptcy and emerging as something completely unrecognizable. HP has itself struggled with round after round of layoffs and a curious future.

Meanwhile, if you believe some circles, there are four new horsemen, this time with Google, Apple, Amazon and Facebook being large players on the Web, with a number of exciting companies playing significant roles in hardware or specialized industries, be it Tesla Motors, Yelp, VMware or Samsung, and established titans like Yahoo!, eBay and AOL repositioning themselves as something new. But holding that position is not guaranteed long term or even short term, as companies must adapt and lead, rather than getting too comfortable.

Change is inevitable, with big picture trends like the Web, the flight to mobile, and increased attention on social and mobile connections making some companies winners and others who miss those opportunites as big losers. Sometimes, a lack of adaptation and innovation can start companies on a negative momentum shift so swift they can never recover. That's business. But it's intriguingly interesting to me how as big life changing events like the dot com rise and crash or the 9/11 attacks fade into something like history, we also forget the companies associated with such a time.

The technologies we use every day were often invented and promoted by people who've passed on and companies that have their own epitaphs. Business plans celebrated and funded by the latest incubators are ones that may have launched and failed with different names and similar ideas just a decade ago. But I'd argue that blazing straight ahead without a knowledge of what's happened before you is dangerous, to both your sanity and your business plan. I am fascinated by Silicon Valley history and have been for the better part of two decades. I fear that the tools we use now provide exceptional access to what's happening today, but do so without context, and to press forward with blinders to the past can only promise that we make the same mistakes that made these once bright companies ghosts of what they once were, if not gone entirely.

History can be dull, yes. But it can also be fascinating. The present, at least most of it, is guaranteed to be similarly forgettable. Maybe instead of documenting our own minutiae, we can empower ourselves with knowledge of one time titans.

While most of the recent personnel news surrounding Yahoo! has dealt with who has left the company rather than who has joined it, the company is expanding its efforts in communities, social and communications, building out a corporate roster below Carol Bartz that features many former Microsoft and MSN alumni, looking to restore the company to its once lofty leadership position on the Web. Joining the purple people on Monday was Steve Douty, one of the first five employees at Hotmail during the Web 1.0 days, tasked with owning product management as the vice president of communications and communities, including properties such as Yahoo! Mail, Yahoo! Messenger, Yahoo! Groups, Flickr and Yahoo! Answers.

Reporting to Bill Shaughnessy, the company's senior VP of global product management, who in turn reports to Blake Irving, Yahoo!'s chief product officer, Douty is responsible for product strategy and roadmap to rollout for the products' current offerings and their future plans, including how they would be monetized.

Like its counterpart at Google, Yahoo! has struggled in some cases about delivering uniformity of product between properties - even while Yahoo! Mail and Flickr especially have maintained top positions in terms of global use. In talking with Douty on Friday, he told me that one thing the company could do a better job with is focusing on integrating these services, and keeping a user longer, aiding them to "go deep into Yahoo and then naturally experience other services in a highly integrated and relevant way", using another Yahoo! property. This hope mirrors a similar challenge experienced by the MSN team during his time there following Microsoft's acquisition of Hotmail at the end of 1997.

Douty, whose Linkedin resume includes stops at Moxie Software and Groopix after Microsoft, joined Hotmail in June of 1996 as the first businessperson at the company, running marketing, product, sales and business development, he said. He recounted to me stories of jump-starting the adoption of free Web-based e-mail through college newspaper advertisements, and making the user experience incredibly simple, including getting a new personal e-mail address within a minute of entering the site and sending a message in the second minute. "In two minutes, you could compose and send a message with your own mail ID," he told me in a face to face meeting last Friday. This led to significant word of mouth for Hotmail that rivaled or even outperformed the service's much-celebrated viral capabilities, including the standard email signature.

Yahoo!'s no upstart, and handful of newspaper ads on college campuses isn't going to turn the trick to get the company back to being recognized as an innovative leader in the Web space. But Douty said he thinks there is significant opportunity for the company to grow from its current position, and capitalize on positions where competitors are not strong.

"There are a lot of incredible assets (at Yahoo!) that have not been adequately stitched and integrated," he told me. "Those assets, combined with a new fresh look on how people use the Internet and interact with others gives everyone an opportunity. There is an opportunity now for a brand-new entrant and a creative set of new offerings for people who are not being served well by anyone today, and Yahoo! is in a position to capitalize on that."

For starters, though Douty is in his first week on the job, it's possible Yahoo! Mail could play a connecting role with the many other services he owns, maybe not as the primary event, but as a conduit to other properties. He recounted Hotmail's challenges with spam, saying Yahoo! and Gmail both do very well at combating spam, and also said Yahoo! Mail is one of the Web's biggest photo properties, simply when photo attachments are included.

Yahoo!'s fight is no longer for search market share against Google. Like Apple's futile OS war against Windows, the time has come to look beyond those initial battles and find new ground to shine. Douty hopes his efforts, and being reunited with many former MSN colleagues, will be key in the company's similar turnaround. Of note, other MSN alumni include both Blake Irving and Bill Shaughnessey as well as John Matheny, the senior vice president of engineering at Yahoo!. With Office and Windows being revenue royalty at Microsoft, MSN never quite got the opportunity to lead in Redmond. In Sunnyvale, Web services are king.

Disclosures: Steve Douty is a director for MenuMax, a Paladin Advisors Group client. He holds the same role for Groopix Inc, a Paladin client in 2009. I am cofounder of Paladin Advisors Group.

The age-old guidance of buying low and selling high has appeared to escape the halls of Yahoo!. If we ignore the Microsoft and Yahoo! acquisition dance of 2008, which ended up in Yahoo!'s staying independent, but losing dramatic market capitalization, search market share, and accelerating an employee exodus, both voluntary and involuntary, it's still clear the company intends to wring all the remaining value out of its vestigial pieces before selling - effectively promising low to no returns on every deal.

Yesterday's news leaked that Yahoo! had plans to kill or spin off a number of Web services, including Delicious, Alta Vista, MyBlogLog and others. Given the abandonment and mismanagement of these properties, this is not too surprising.


Alta Vista's mismanagement goes back more than a decade. In 1999, I wrote a story (this is pre-blog, so I'd love to say "post") on Internet Valley's Web site about how Compaq was not leveraging Alta Vista after they acquired Digital Equipment Corp. (See: Warning: Compaq is On the WWWarpath) The site is a shell of its former self, once being my go-to search site.

But Delicious is what has everyone up in arms, as the company was among the first Web 2.0 darlings, and is still widely used, as the Web's definitive bookmarking site. I've personally tagged more than 3,000 items in the last few years, most of them capturing external links to the blog. Yahoo! has managed to take a site with a deep bench of fans, considerable use, and make it obvious they would love to give it away. It's not even clear if anybody really works on it any more, or if it's just being maintained by the company's techies.


On this background, both Web traffic stats and sharing statistics from third party sites show the party long since ended at Delicious. Compete, whose traffic graphs are anything if not suspect, still manages to display some trendlines right, showing a decrease in traffic of nearly one half in the last year. In parallel, AddThis, the social sharing site, shows decreased Delicious bookmarking, down by 14% in the last three months, the fastest declining site of its kind on the service.


The moves Delicious has made have been odd or delayed. The long-awaited "New Delicious" was way behind schedule, and recent bumps made sharing to external services even more difficult. It's as if the company wanted to punish the users it had left.

If you want to get a fantastic return on a sale, the best time to sell is when you don't have to, when things are going fantastic and you have real momentum. Yahoo! has essentially telegraphed to the world that it wants to ditch Delicious to someone, anyone. But the catch is, its users already started ditching it themselves. Buy high, kill.... sell. That's the wrong order of things, Yahoo!.

20. The development teams behind Google Reader and Blogger are unified following a reorg, under a new project called "Google Breeder".

19. The Google New project accidentally gets uploaded to the wrong set of servers and overwrites Google News with internal company project updates.

18. Twitter becomes so successful that long-form writing is considered anything which is between 135 and 140 characters. As a result, Google runs out of interesting things to index.

17. Google's goats get an agent, defect to Zynga to promote Farmville.

16. Google's book indexing project fizzles as its realized nobody actually reads any more.

15. Due to an impasse with broadcast networks and cable's fear of Google's growing impact, Google TV traction stalls as only commercials are available.

14. With the founders of Wave, YouTube, AdMob all leaving or changing roles in the last weeks, leaders of all project teams within Google dust off their resumes and post them as shared Google docs with high-priced executive recruiters.

13. Aggressive legalities determine that Google has to take another photo of your home on StreetView if you don't like the current one.

12. Somebody actually asks a question to Larry Page or Sergey Brin about the future of Google Knol or Google Base in a public forum.

11. The code base between Google's new robot cars is mixed with the Android code base, making our phones autonomous beings that call and leave messages with text to speech to anyone in our address book whenever they want.

10. Google Buzz is discontinued after unfavorable press coverage. The Lively team is persuaded to come back, and is given the keys to "social".

9. Eric Schmidt loses a bet to Carol Bartz, is forced to buy Yahoo!. The new company is called Yahoogle. It passes anti-trust scrutiny somehow.

8. Yahoo! is determined to own the patents to Pay Per Click following their acquisition of Overture, Google forced to remove PPC from their offerings.

7. AJAX is the new Flash.

6. Jimmy Hoffa is found buried in Google's famous ball pit.

5. Microsoft cracks the code behind Google's search algorithm. Using SEO, Bing results flood Google's index for all terms.

4. An overzealous team of programmers looking for career advances takes Eric Schmidt's off the cuff remarks seriously, embarks on setting up programs to help people move and change their names at age 18.

3. Paul Buchheit is determined to own the intellectual property and trademarks to Gmail and its underlying code. Thus, Gmail and its users become property of Facebook.

2. Apple's suit against HTC is successful, freezing the development and deployment of Android for all partners.

1. It is revealed that the same 50 people click on 99% of AdSense ads, and most of them are marketers just demoing the product to potential clients.

It's Monday, which must of course mean it's Twitter new employee day (TM). The company, which is continuing its expansion practically every week with new hires, onboarded three more today, including the lead Web developer for Yahoo! Messenger, focused on scalable Web technology, a senior database admin from Bebo (who also had history at Yahoo! and Apple), and a software developer who hails from the unlikely source of the Indianapolis Star, with background as an airplane pilot.

Twitter's growth brings the company's "Team" list to 176 total members, including contractors and other contributors to the company.


Adrien Cahen, formerly a senior front-end engineer at Yahoo!, was the lead Web developer for Yahoo! messenger, until announcing his resignation after 3 1/2 years at the Sunnyvale company on March 15th. In the resignation announcement on his blog, Cahen said Yahoo! had a great company culture with cutting edge technologies under development, and said his efforts on highly used products was a "very humbling experience."

One of the products he most recently contributed to was the launch of Yahoo! Messenger 10, and its inclusion of insider pages, mini versions of the Yahoo! homepage built on YQL (Yahoo! Query Language). He can be found at @gaarf.


David Bravant, the DBA joining Twitter by way of Bebo and Yahoo! spent nearly six years at Apple from 1999 to 2005, and doesn't have a long history of using Twitter, having possibly just opened up his Twitter account today, with no activity yet. (See: @clarethammer)

Dana Contreras, the pilot hailing from Indiana, most recently of the Indianapolis Star, describers herself as a software developer with a wide array of platform experience, and lists her new job as software engineer on Twitter's platform. Dana is also the developer behind JLex.org, a Japanese language reference and learning tool, with integrated kanji search. She can be found at @danadanger.

Today, Todd Kloots (@toddkloots) joined Twitter after more than seven years at Yahoo! as a front-end engineer focused on the Yahoo! User Interface (YUI) library for interactive Web applications, after previous work on the company's massive Yahoo! Mail product, where he worked on the service's front-end and design. Todd's joining Twitter continues the company's growth, pulling like a massive magnet from Silicon Valley's top companies toward the blue-logoed beacon in San Francisco, following hires from Facebook, Google, and other Valley leaders.



Kloots, who is credited with 10 years of web dev experience, joins Twitter a week or so after Michael Busch (@michibusch), who worked at IBM for the last five years as a software engineer and architect, joined the company as a search engineer. His LinkedIn profile says he was focused on "indexing, search and scale-out" for IBM's eDiscovery Analyzer product. As mentioned frequently here, Twitter's mountain of data badly needs some strong attention toward indexing and scale-out, so his hire looks to bode well for making their search tools more relevant.


Jesse Stay and others highlighted Busch's hiring last week. Also noted last week was Twitter's crossing the 140th employee mark, a milestone for the company obsessed with 140 characters. (In the comments, a fellow Twitter employee says Michael was not #140) Jesse also highlights Busch's background with the open source Lucene search project.

You can see Kloots' presentation of the new YUI 3 at Yahoo! from the fall of 2009 at the Yahoo! Developer Network for an idea of his previous work.

Internal code names for tech projects in Silicon Valley and beyond are a celebrated pastime - ranging from the predictable alphabetically-ordered release numbers to fanciful references to family members, vacation spots, animals or practically anything that doesn't actually describe the product itself. (See Apple code names or Microsoft code names for examples)

The code name for Google's new Buzz product? Taco Town. Not a grandiose reference to composers long since passed on, or beautiful vacation spots around the world, as other companies have done, but instead, an ode to good-tasting greasy food.

While Google obviously thought "Buzz" was a better name for their new aggregation play than Taco Town, despite archrival Yahoo! having used the name first, you can see tacos as part of today's launch, with a delicious-looking taco gracing the "Just the good stuff" description on the main Google Buzz page, and, throughout the many screenshots and demo video, the individual's name read as "Ted Taco".

If It's A Taco, It Must Be Good Stuff

The Notorious Ted Taco from Google Buzz
So where does Taco Town come from? Assuming Google didn't name it after one of the restaurants that dot the US, including establishments in Scottsbluff, Nebraska and Carthage, Missouri, the Taco Town name hearkens back to a satirical commercial from Saturday Night Live, called "Taco Town", where Bill Hader, Jason Sudeikis and Andy Samberg go out for Mexican food, only to find their delicious dish to be a challenge, with layer upon salivating later being added to the entree.

The announcer, introducing the new taco at Taco Town, teases with his description:

"We take a crunchy, all-beef taco, smother it in nacho cheese, lettuce, tomato and our special southwestern sauce. Then we wrap it in a soft, flour tortilla with a layer of refried beans in-between..." adding on... "Then we wrap that in a savory corn tortilla with a middle layer of Monterey Jack cheese..." and later... "And it gets even awesomer, when we take a deep-fried gordita shell, smear on a little of our special "guacamolito" sauce and wrap that around the outside...", "But it gets even bigger! Because we bake it in a corn husk filled with pico de gallo, then then wrap that in an authentic Parisian crepe, filled with egg, gruyere, merguez sausage and Portobello mushroom...." followed by more layers including a Chicago style deep dish meat lovers pizza, rolled in a blueberry pancake, fried golden brown, and served with "spicy vegetarian chili".

As the faux commercial says, " It's 15 great tastes all rolled into one." You can see it embedded below, provided Hulu supports your country.


With one's social graph and social activity fractured asunder all over the Web, you can see why Google would want to roll all these great tastes together into one product, Buzz. The company also likely sees the difficulty in taking our many diverse activities, social graphs, and privacy concerns as different layers that must be combined to deliver a compelling offering.

Regardless of what Yahoo! and Microsoft might think about the new service, or its name, I think we can all agree it looks a lot better than "Google Taco Town".

Facebook Up Slightly, MySpace and Twitter Flat to Down
Despite November being nearly half over, the monthly traffic statistics from October have just been released by Compete.com, and it looks like there are no major surprises in the social networking arena. Despite the recent improvements and continued hype, traffic to Twitter.com decreased slightly, by 2 percent, month over month, tracking at the level it saw in June of this year, and lower than the previous three months. Facebook, the #3 site overall worldwide, behind only Yahoo! and Google, climbed more than 3 percent, to almost 129 million, while MySpace stayed steady around 50 million unique visitors (15th overall).


FriendFeed and Posterous Decline - While Twine Plunges
Where one saw more movement was in the lower tiers, as FriendFeed continued its descent following the Facebook acquisition, shedding nearly 7 percent of visitors, dropping below the 700k mark, from a one-time peak above 1 million, and Posterous dropped more than 12 percent, showing just under 1.2 million visitors. Twine, which once peaked above 2 million, is now just over 120,000.


Yahoo!'s Slow Decline Comes as Facebook Rises Toward the #2 Spot
Facebook's slow but steady growth actually has them looking less in the rear view mirror, toward companies like Twitter (who scored 23 million uniques vs. Facebook's 129 million) and more at the big gun right ahead of them - Yahoo!, which continued its slow descent, dropping just over 1 percent, to 135 million unique visitors. In fact, one more month with the same trajectory would have both networks tied at about 133 million visitors, so we could see a change in placement come November.


Google's Position at #1 Remains Unchallenged (Shown With YouTube)
Unsurprisingly, Google reported in at #1, again, counting almost 150 million unique visitors in the month, according to Compete (which in my opinion is probably low). In addition, the company's YouTube subsidiary tracked just under 85 million unique visitors, good enough for the #5 position worldwide on its own. GMail continued its climb to another 9.3 million visitors, up 98% from this point last year.

Surprisingly, GMail's position is more than 3 times higher than that of Hotmail.com, which has even been surpassed by Apple's Me.com MobileMe e-mail offering. Me.com sported 3.5 million visitors, growing 98% year over year, contrasted with Hotmail's 2.5 million, which decreased 7 percent, according to Compete.


LinkedIn Stays Hot - See Versus Twitter
Interestingly, during the recession, with high unemployment, LinkedIn.com traffic increased 3.3 percent in the month to 15.5 million unique visitors, up 89% on the year. Monster.com, the massive job site, tracked in with 41.5 million unique visitors, good for #20 in the world, up 47% on the year.

Some other sites of note:

• Apple.com traffic tracked at 21.4 million, compared to 15.5 million for hp.com and 13.4 million for Dell.com.
  
• Digg.com traffic decreased less than 1 percent, up 57% on the year, good for 43 million uniques.
  
• Technorati.com traffic was flat, with only 2.8 million unique visitors.

Disclaimer: Compete statistics are known to be imperfect, but they are always interesting.

That Twitter is dealing with massive amounts of data flowing through its servers these days would be an understatement, as the service sees strong growth and significant mindshare. With the company having passed what looks to have been its rockiest struggles over the last twelve months, Twitter is now getting to focus on rolling out some significant new features, from Lists to geolocation, trend definitions and retweets. But the microblogging giant looks like it is taking extra steps to harness the power of its rapidly-expanding data set.

If the company's own team list is to be believed, they just picked up Utkarsh Srivastava, a highly respected senior research scientist at Yahoo!, who is best known for his work on building large-scale distributed systems, specifically his efforts with Hadoop.

Hadoop, similar to the Google File System, is a framework that enables applications to work over distributed server nodes and significant data sets - potentially ranging in the petabytes. Yahoo!, Google's off and on competitor, has been the company most associated with Hadoop. While at Yahoo!, Srivastava was one of the original designers of "Pig", an Apache project for analyzing large data sets, which leveraged Hadoop. (See also the research paper: Pig Latin: A Not-So-Foreign Language for Data Processing)

Srivastava, a PhD graduate from Stanford University in Computer Science, has been working at Yahoo! Research since 2006. (See his home page and LinkedIn profile)

Not knowing what aspects of Twitter Srivastava may be working on, it's premature to assume whether his efforts will be primarily focused on new initiatives, or simply helping the company scale its growth. I can dream and hope that he can be the missing piece that brings Twitter's high potential search engine fully online, but that is no doubt a big project indeed.

Update: This hire has been confirmed by Srivastava and also covered by TechCrunch.

Prior to the debut of Apple's iPhone, software developers wanting to make miniature applications to reach Macintosh users had a direct route to customer's desktops through the development of Mac OS X Dashboard widgets. Billed as major functionality debuting in Mac OS X 10.4 (Tiger) in early 2005, Dashboard, much like Konfabulator before it, featured widgets including weather forecasts, stock updates, calculators or simple utilities, like a dictionary or thesaurus. Over time, developers managed to make a number of creative uses for these apps, from delivery updates to flights' status, or even lightweight arcade games.

But four years later, Dashboard is dormant, while not yet completely dead. A quick calculation of the total number of widgets listed on Apple's Web site is just over 4,500, of which 820 are international. In remarkable contrast, Apple's iTunes Store loudly proclaims its haul to be more than 75,000, of which a massive 1,394 were posted just last Friday.

Want to know how many new Dashboard Widgets were posted last Friday?

Zero.

Interestingly, Apple's Dashboard Widgets site highlights the last 50 "Just Added" to the Web site, and for whatever reason, between September 9th and September 21st, no Dashboard Widgets were posted. Maybe the one guy whose job it is to get them approved was on a two-week vacation?

And in the week, starting with Monday, only 39 total Widgets made it into into the directory, including fascinating titles such as the "Iowa Hawkeyes Football Schedule" widget or the "Countdown to Thanksgiving" widget.

With iPhone app developers having a fast-growing audience of millions ready to spend real money on applications for their mobile phone, the idea of creating a miniature application for the desktop, for free, probably doesn't have much pull. It's no secret that the iTunes App Store's runaway success has played a big role in making any discussion about Dashboard completely unnecessary.


The louisgray.com Widget as posted in 2007
Just a few years ago, many thought widgets were going to be the next hot thing in desktop apps. Konfabulator was purchased by Yahoo! and turned into Yahoo! Widgets, which claims nearly 6,000 desktop widgets. And way back in 2006, I even made a dashboard widget for louisgray.com, which I later updated in 2007 and still works. But the talk of widgets faded, as people primarily chose to use the Web browser and their iPhones as the conduits for Web data, preferring not to have a gazillion little widgets floating about their screens.

Considering the dramatic drop-off in buzz around Dashboard widgets, and a corresponding meteoric rise in iPhone deployments and penetration, it's no surprise to me that you see this gulf, which now measures almost 20x in favor of iTunes applications. It's enough to wonder if Apple will ever bring up the Dashboard again, except maybe to provide a place where iTunes applications purchased for the iPhone today can sometime live. They won't kill it outright, but it sure looks like a dinosaur, after only four years.