January 30, 2018

Real Valley Stories: Search Marketing With the Open Directory Project

Editor’s Note: Part 12 in an irregular series of stories from my many years in Silicon Valley. Part 11 talked about the time I got called into HR's office to meet with lawyers over industrial espionage. This time, a story involving gray hat search engine marketing in the early days of the Web.

DMOZ is now closed. 

Believe it or not, before the world of automated spiders that crawled the entire Web and ranked the results for your searches, much of the way we found content on the Internet was thanks to manual updates from an invisible army of directory editors. Yahoo! defined the initial dot-com era, with its hierarchical oracle making or breaking traffic downstream, as sites were organized and shuffled into categories by unseen text tweakers, much like the editors of Wikipedia try and keep its tens of millions of article pages up to date, with a seemingly fluid mass of edits to keep the live encyclopedia current.

But Yahoo! wasn't the only Web directory. Rich Skrenta and others, also behind Web 2.0 efforts Topix and Blekko, introduced the Open Directory Project, referred to as DMOZ, for it was hosted on the directory subdomain of the Mozilla.org site, with an objective of harnessing contributions from around the world (like Wikipedia), to build a directory, similar to Yahoo!, that could plug into any site that wanted to host one. In a time when many sites were seeking Internet traffic and riches by acting as the front page for the Web, attaching the open directory project to your portal could give you an edge and not require you to bring on a ton of staff.

As with Yahoo!'s directory, a company's inclusion in the DMOZ directory could act as a binary gate as to whether or not potential users would find you. In 1999, working as a Web marketing manager for a Web startup that offered internet faxing and conference calls, I found myself irked to see that our services were not included in DMOZ. Making things worse, the categories I would expect to see us listed in seemed slapped together and without an official owner. Given my understanding of the space and knowing our many competitors, I registered an account and requested to moderate the relevant category.

The DMOZ Internet Fax Listings from 1999 (via Archive.org)

Not too long afterward, I was given the option to update the category, including those of our competitors. As it was nearly two decades ago, I honestly don't remember if I used my company email or a Yahoo! email or equivalent, but I didn't try and disguise where I worked. My application had gotten through.

FaxCube from the year 2000.
When I did log in, I found the content in a state of abandonment. There wasn't much you could do with a site's listing. Give it a title, a link, and a short description, literally about a dozen words. It was fairly impossible to differentiate services from one another, especially in a commoditized space where the core function was pretty straight forward. But I cleaned up the descriptions for all the entries, including our competitors, to accurately display their offerings. Some offered email to fax services, while others offered the reverse. Some offered broadcast faxing. Some required a proprietary fax viewer, and others were completely Web based. That kind of thing.

When content was edited in DMOZ, edits would later propagate downstream. Sure enough, my colleagues noticed a spike in Web traffic to our main sites, with referrers coming from all the places DMOZ was set up. For no cost, I had a clear impact in our customer acquisition funnel, and maintaining the DMOZ became part of my ongoing work.

Later, DMOZ added the option to highlight two entries in the group, which were solely up to the moderator. This, of course, gave me the option to expand from a gray area, to clearly going over the line into promotion. It went without saying I thought our service was the best, and highlighted it at the top. I also chose to highlight a partner site (the Netscape Fax Center) that was white labeling our service, essentially the 1 and 1A positions.

This got even better when we soon realized DMOZ was fueling search results for AOL. The more people who searched for Internet Faxing on AOL, the better we were to see results.

DMOZ listings for Internet Telephony in early 2000. (Via Archive.org)
In early 2000, we introduced a new Web based conference call and meeting product. Addicted to the free traffic from DMOZ, my team asked me to see if I could get our content similarly included on whatever the equivalent was for Internet conferencing. I poked around, and, again, applied to be an editor for the Internet Telephony page. It wasn't a perfect fit, but it was pretty good.

Again, I got approved, and again, I added our sites, and again, I promoted our main site and a partner site to be included as the pair at the top of results. Again, we started to get plenty of Web traffic from DMOZ and its downstream partners, accounting for more than double digit percentage of our traffic to both properties. But this time, it wouldn't last long. Maybe it was the clear marketing copy promoting both services, but one day, I logged in and saw our service demoted to the standard results, with another in its place. I reversed the change, and it wasn't too long until there was a debate upstream as to whether these listings were in good faith.

PhoneCube, in all its glory, from 2000.
Not too long afterward, my moderation rights had been removed from both sections. I'd essentially been fired from my side job of gray hat SEO, long before most of us knew what that meant. And yes, my engineering peers lamented the loss of traffic, as our more organic listings, on AOL search and elsewhere, didn't carry as much weight, once we became one of the fold.

Had I just stayed content to include our services in the listings, or even just to stay a moderator of the less exciting Internet faxing space, it's possible I could have been editing text descriptions for our sites and our competitors indefinitely. But pushing our own products and our partner sites turned out too good to be true. The "Wild West" Internet traffic rush didn't last forever, and, as it turns out, neither did our products. I was laid off after we couldn't get a funding round closed at the beginning of 2001, and a few months later, the company was parceled off to become part of Oracle (see 2006 post). Some of my colleagues still work there nearly two decades later. As for DMOZ, it too shut down a few years ago, a relic in the world of Google and Facebook.

Disclosures: This whole post is a disclosure! I worked at 3Cube during this time. I work at Google now, a perceived competitor to whatever Yahoo! and AOL call themselves now. And as I work on Google Analytics, this is not a post about SEO to all you SEO/SEM folks.

January 18, 2018

With Web at the Core, Chromebook Options are Strong, Plentiful

This looks like an ad. But it's just a few recent Chromebooks.

In 2011, on my first day at Google, I was asked to pick out a laptop. The choices were slim - a thin Apple MacBook Air or the larger MacBook Pro, a forgettable Windows equivalent, or a Linux device more suitable for engineers. While I had the company's first foray into Chromebooks, the CR-48, at home, in addition to my own personal Mac, picking a Chromebook wasn't even an option. The Web-centric OS, which focused on keeping all data in the cloud, and leveraging Web apps, wasn't ready for my every day use.

A few months later, I ran into then SVP of Chrome Sundar Pichai, in the office stairwell as we were on to our respective meetings. Pointing to my MacBook Air, I told him I couldn't wait to turn it in and go completely ChromeOS at home and at the office. In his usual humble and understated way, he said the team was working on it, and to stay tuned. Not too long afterward, in another unplanned hallway conversation, he introduced me to a VP on his team developing hardware, and offered me up as a willing beta candidate.

The 2013 Chromebook Pixel (version 1)
I didn't think much of the choice encounter until early 2013, when I saw Sundar take the stage and unveil the Chromebook Pixel, a high-end Chromebook with a touchscreen, and promised faster speeds and memory.

As I recounted a few years ago on Google+, I saw Sundar as available on IM shortly after the event and congratulated him on the exciting launch. His IM came moments later... "Do you have one yet?" Surprised, I said I didn't, and it was no big deal. I had no such illusions of self-importance. But he answered directly, "I'm so sorry. You were supposed to be on the list." Fast forward, less than an hour later, I had a brand-new Pixel - and I haven't seen a need to use a Mac since.

That a Googler is using a Chromebook isn't newsworthy, obviously. Water is wet. But I remember a time when betting on a Web-centric device like a Chromebook was a real leap of faith. There were always excuses not to make the switch, be it a specific piece of software, some concern about printing, or general distrust of the unknown. Maybe we were worried about moving local storage to the cloud, or editing photos, or losing access to some premium software on Mac or Windows we'd already paid for - often at a cost even higher than a new machine.

Chromebooks have proven exceptionally popular in schools, thanks to their versatility and low cost. And as people become more mobile-centric, their data also becomes more portable and Web centric. Just as you expect to have your data follow you from phone to phone, moving from device to device should be seamless. Like I'd said in 2012, the future of local storage is practically none at all.

This summer, I got my wife a touchscreen convertible Chromebook for less than $100.

Watching the many different options for Chromebook hit the markets feels a lot like the same momentum we saw when Android's many partners took imaginative approaches to new handsets. While we essentially knew the rigid details coming from Cupertino for both computers and phones, Google partners built big and small and with any number of differences to set each apart, from brands as diverse as Acer, Asus, Dell, HP, Lenovo, Samsung and Toshiba.

Now the decision process is one of plenty, not scarcity. So many options, pretty much all of them good. You can get small screens or big screens. You can get touchscreens and convertibles that act like a tablet. You can run Android apps, or even mark up the screen with a digital pen. All very cool.

This summer, while on a family vacation in Chicago, after seeing so many positive reviews for Samsung's Chromebook Pro, I figured it was time for an upgrade from my two year old Acer 710. I quickly bought one on Amazon, had it delivered to an Amazon Locker down the street the next day, and after entering my Google credentials, I had made an incredible upgrade, with no data migration needed. It was almost too easy. (And yes, that's the laptop I'm on now)

Having seen the Pixelbook, the successor to that 2013 Pixel and its 2015 follow-on, and even more good reviews, I'm already getting that itchy feeling and have added the newest device to my shopping cart more than half a dozen times, desperately wanting it, but knowing the Chromebook Pro has a long life left.

Meanwhile, my wife's slim Asus Chromebook I picked up this July gets constant use, and my 9 year old twins bang away on inexpensive Acers to do work in Google Docs. They do the work of machines that cost 10 times as much, without coming bundled with the worry that you lose your data in the event of a disk failure. And the gaps that may once have been there in 2010, 2011 or 2012... they're gone.

If you're an elite creative software wizard who has a custom setup, then keep it up, but for the rest of us who use our devices to create, engage, consume and share, the Web is the most powerful device there is, and Chromebooks were designed for it. They've come a long way.

Disclosures: I work at Google. You knew that. But I still pay for my Chromebooks, except for those provided for me to use at work, obviously.

January 17, 2018

Smartphones Have Virtually Eliminated Boredom from the Modern Life

People are constantly on their phones. All day.
There's a flurry of debate over whether smartphones and their apps have become too addicting. While there is no complete agreement over how often smartphone users access their phones each day, estimates put the number at anywhere from 80 to 150 times. If you're a typical human who is awake about 16 hours a day, that's five to ten accesses per hour. Every hour. You might even put your own estimate much higher, or, instead, see it as one long continuous touch that consumes the entire day.

Independent of the discussion of whether this is a "good thing" or not, the ability to constantly engage with one's phone, checking messages from different apps, getting the latest news instantly, window shopping or achieving a new high score, the device has virtually eliminated the opportunity to be bored - acting as the glue that connects times when you're otherwise active. The smartphone acts as a space filler and a constant alternative for whatever else you might be doing.

Not too long ago, there was something we recognized as a quiet space between activities. Mental breaks. Whether that was standing at a corner for the light at the intersection to go green and allow us to cross the street, or taking an escalator at the mall, or waiting for the bus, we recognized those gaps as something like boredom. Was there nothing on TV? Bored. Forced to wait in line at the supermarket? Bored. Finished your book? Bored. Is the baseball season over? Bored for four months.

Think people aren't constantly on their phones while driving? Think again.
But this isn't the case now. If you look around at people, everyone is seemingly in a state of constant engagement with their phones. Drivers at intersections waiting for red lights to turn are waiting for cues from the cars next to them to indicate the signal has changed. Pedestrians are walking with their feet slightly askew to avoid unseen stumbles, and draft behind the people ahead of them, one hand holding the phone at an angle, looking up every few steps for potential surprise. Those waiting for the bus only interrupt their phone use to glance up and see if their ride is on its way.

Many a word has been spilled about how smartphones have invaded daily lives. Couples go to restaurants and read their phones instead of talking to one another. Colleagues may glance at their phones and tweet while you're talking to them, looking up on occasion to see if whatever you're saying is more interesting than whatever popped up on their screen. It's no longer a challenge to find something to do. Instead, it's a battle to see who can be the most sensational or carry enough weight to trump the alternative that is constantly available on a 4 or 5 inch screen.

Often, when presenting to rooms full of people at events, I see attendees on their phones. It's been years since you could authoritatively demand a 'laptops down' meeting and expect to get everyone's full attention. That people are going to try and deliver continuous parallel attention is a reality, and you are in a constant battle to earn their mind share, in a hope that your engagement will be more lasting and more significant than what their phone has selected to bring to the fore.

In the last decade plus, more of what we used to depend on full sized computers, cameras, televisions, maps and more has been miniaturized and made portable in our pocket. This has allowed our entertainment, learning and communications machines, our commerce engines... to be constantly with us. People meet their soulmates on their phone. They get paid on the phone. They can order food and have it delivered, all from their phone. If life's every important value, to consume, to share and to survive, can be designed and managed from your phone, it really begs the question of whether the world within the screen is less valuable than the one on the outside of it.

My own kids, still under 10, don't yet have phones. It's already challenging as a parent to provide them the structure they need to learn independently and prioritize work over entertainment, without giving them a magic device that does it all in their pocket constantly. But they do know they live in a world where boredom is a practical impossibility, and where everything is practically a request away.

Life moves faster now, and it isn't boring. That's no longer an option.

January 10, 2018

Linking Less and Talking More: Disappearing Web Mentions


The World Wide Web was designed to primarily do three things - inform, discover and connect. A globally connected series of documents could instantly bring you to the thoughts and experiences of someone across the world. In the earliest designs of the Web, it was through hyperlinks that you would find those new voices. Links brought you new sources of data, and those downstream documents led you even further to new people and ideas.

As the Web evolved, and incorporated photos, videos, streaming, and all manner of media, discovery expanded to include search. Without an explicit link, you could still find pointers to new content in the results of your query. Destination sites, acting as content hubs, would surface new content, usually within their network, of recommendations you might like. Ads, essentially links with pretty pictures, would offer another exit.

WebCrawler: One of the Web's first search engines
When blogging was the main medium of first person information sharing, prior to the rise and later domination of real time social streams, the way we discovered new voices was through links to others. I'd mention those I agreed with and highlight, with more links, those I didn't. One popular feature in practically everyone's sidebar was a blogroll, to show those with closest ties or just who we liked to read. And there were custom search engines, like Technorati, which when combined with tools like Google Alerts, could let you know when somebody mentioned you on the Web.

Technorati: The original blog and link search engine
But over time, a number of things happened to chop away at this fluffy cloud of friendly discovery.

1. Many Blogs Gravitated Toward Internal Linking, Not External Linking
The big sites realized that keeping visitors on their own site was more profitable and aided their metrics more than sending them away did. And while there is obvious irony in my posting to my own discussions on this from the past, we actually had lengthy discussions about these internal linking practices in 2007 (Part two and part three
Arguments a decade ago in favor of internal linking were that site visitors were familiar with companies and topics discussed, and could see previous coverage by their publication to learn more if they weren't. And any link off site started feeding the ad revenues of a potential competitor.
2. Dedicated Blog Search Sites Didn't Graduate to Quality Businesses
Technorati was a specialized blog search engine that skipped the general Web and went directly to blogs for its content. Its leaderboard of bloggers was closely watched, as were trending topics on the site that led to see what the blogosphere was discussing. But it was seemingly always in financial trouble, and has pivoted beyond recognition to whatever it is now.  
A 2010 interview I had with the company's leadership team claimed a pivot to quality, but their CEO was gone a year later, and so are pretty much all the discovery tools that initially aided me to find some of the best voices of the Web 2.0 era. And yes, Google Blog Search quietly disappeared not too long afterward.
3. Blog Discussions Pivoted to Real Time Streams and Sharing
As I noted in 2009 (yet another internal link, am I right?), linking between communities was declining in favor of retweets on Twitter or sharing into the stream. The microburst of a little site traffic would provide that one time dopamine hit, but not leave a trail for later web spiders to find.
My top referring sites, via Google Analytics, from previous years

As the social streams of Twitter and Facebook took over, and bloggers (me too) got distracted, the share became the canonical mention. Your mentions on Twitter, or your notification of shares on Facebook, were faster delivered and easily quantifiable. And individual profile owners are quite unlike the publishers looking to deep dive into their analytics to discern where traffic came from.

Today, Brent Simmons laments the result of all the mentions going to the streams and leaving the Web. With links decreasing, and blog search being a relic, he yearns for a way to find again how his business is mentioned on the web or when he is being linked to. Joe Gregorio, like me, wistfully remembers analysis of referral logs to find how people found you... all through links.

Simmons' proposal is a limited one, keeping it to the MacOS/iOS community. A small project like that shouldn't be too challenging, but it speaks to a bigger problem, where the connections we once demanded are an afterthought behind the latest viral tweet and trending Facebook share. Streams are ephemeral, but the Web was built to last. It'd be great to see new voices building, informing and connecting again.

Disclosures: I work at Google and am on the Google Analytics product. So I think about publisher tools and visitor stats more than most.