You Are Browsing The Rant Category

Google Removes Related Searches

April 19 2013 // Rant + SEO // 45 Comments

This morning I went to use one of my go to techniques for keyword research and found it was … missing.

Related Searches Gone

Related Searches Option Gone

It was bad enough that the new Search tools interface was this awkward double-click menu but I understood that decision. Because most mainstream users don’t ever refine their results.

But to remove related searches from that menu altogether? In less than a year related searches went from being a search tip to being shuffled off to Buffalo?

WTF!

Out of Insight

Clooney is Pissed

Google needs to understand that there are SEOs, or digital marketing professionals if that makes it easier, who are helping to make search results better. We’re helping sites understand the syntax and intent of their users and creating relevant and valuable experiences to match and satisfy those queries.

I wasn’t happy but wasn’t that upset when Google introduced (not provided). But as the amount of (not provided) traffic increases I see no reason why Google shouldn’t implement my (not provided) drill down suggestion. Seriously, get on that.

But then Google merged Google Trends with Google Insights for Search and in the process removed its most useful feature. That’s right, knowing what percentage of the traffic that was attributed to each category let SEOs better understand the intent of that query.

Now Google’s taking away the interface for related searches? Yeah, you’ve gone too far now. Hulk mad.

Stop Ignoring Influencers

You Wouldn't Like Me When I'm Angry

Just like the decision to terminate Google Reader, Google doesn’t seem to understand that they need to address influencers. And believe it or not Google, SEOs are influencers. We’re demystifying search so that sites don’t fall for get-rank-quick schemes. And you need us to do that because you’re dreadful at SEO. Sites aren’t finding much of your educational content. They’re not. Really.

In the last year Google’s made it more and more difficult for SEOs to do good work. And you know who ultimately suffers? Google. Because the content coming out won’t match the right syntax and intent. It’ll get tougher for Google, over-time, to find the ‘right’ content and users will feel the slow decline in search quality. You know, garbage in, garbage out.

Any good marketer understands that they have to serve more than one customer segment. Don’t like to think of SEOs as influencers? Fine. Call us power users and put us back on your radar and stop removing value from the search ecosystem.

No Such Thing As A Good Scraper

March 14 2012 // Rant + SEO // 24 Comments

I have 155 pending comments right now. The overwhelming majority of them are pingbacks from benign scrapers. Some may see this as a boon but I view these scrapers as arterial plaque that could ultimately give the Internet a heart attack.

Here’s my personal diagnosis.

The Illness

My definition of a benign scraper is a site that scrapes content but provides attribution. I’ve gotten a ton of these recently because of links I received in high profile sites within the search community. Those sites are the target of these scrapers so my link gets carried along as part of the deal.

Benign Scraper Pingbacks

The attitude by most is that the practice won’t damage the scraped site and may actually provide a benefit through the additional links. Heck, Jon Cooper at Point Blank SEO even came up with a clever way to track the scrape rate of a site as a way to determine which sites might be the best candidates for guest posts.

Signs and Symptoms

But what do these scraper sites look like? Some of these scrapers might have original content mixed in with the scraped content but in reviewing my pingbacks this seems like the exception and not the rule. Most of these benign scrapers are just pulling in content from a number of feeds and stuffing it onto the page hoping that users show up and click on ads and that the content owners don’t take exception.

Benign Scraper Attribution Example

“Hey, I gave you a link, so we’re cool, right bro?”

No bro, we’re not cool.

This stuff is garbage. It’s content pollution. It is the arterial plaque of the Internet.

The Doctor

Google is trying to keep up and often removes this dreck from the index.

Benign Scraper Deindexed

But for every one that Google removes there’s another that persists.

Benign Scraper Indexed

How long until the build up of this arterial plaque gives the Internet a heart attack? One day we’ll wake up and the garbage will be piled high like a horrifying episode of Hoarders.

Support Groups?

The industry attitude toward these scrapers is essentially a tacit endorsement. It brings to mind the quote attributed to Edmund Burke.

All that is necessary for the triumph of evil is that good men do nothing.

We turn a blind eye and whistle past the graveyard happily trusting that Google will sort it all out. They’ll make sure that the original content is returned instead of the scraped content. That’s a lot of faith to put in Google, particularly as they struggle to keep up with the increasing pace of digital content.

Are we really this desperate for links?

Desperate for Links Example

Yet, we whine about how SEO is viewed by those outside of the industry. And we’ll whine again when Google gets a search result wrong and shows a scraper above the original content. Indignant blog posts will be written.

Treatment

Even if we wanted to, we have few tools at our disposal to tell Google about these sites. The tools we do have are onerous and inefficient.

It doesn’t have to be that way.

Why not build a Chrome extension that lets me flag and report scraper sites? Or a WordPress Plugin that lets me mark and report a site as a scraper directly within the comment interface. Or how about a section in Google Webmaster Tools where I can review links?

Sure, there are reporting issues and biases but those are solvable problems. Thing is, many doctors have a God complex. Google may not think we’re able to contribute to the diagnosis. That would be a mistake.

Cure?

Disaster Girl Dares You To Ignore Scrapers

Maybe we don’t want to be cured. Perhaps we’re all willing to let this junk persist, willing to smile as your mom finds one of these sites when she’s looking for that article you wrote. Willing to believe that your brand is totally safe when it appears on these sites. But the rest of the world isn’t nearly as savvy as you think.

I know many of these links work, but they shouldn’t. The fact that they do worries me. Because, over time, people might not be able to tell the difference and that’s not the Internet I want.

Today these scrapers are benign but tomorrow they could turn malignant.

Delicious Turns Sour

December 19 2011 // Rant + Technology + Web Design // 8 Comments

In April, the Internet breathed a sigh of relief when Delicious was sold to AVOS instead of being shut down by Yahoo. In spite of Yahoo’s years of neglect, Delicious maintained a powerful place in the Internet ecosystem and remained a popular service.

Users were eager to see Delicious improve under new management. Unfortunately the direction and actions taken by Delicious over the last 8 months make me pine for the days when it was the toy thrown in the corner by Yahoo!

Where Did Delicious Go Wrong?

Delicious Dilapidated Icon

I know new management means well and have likely poured a lot of time and effort into this enterprise. But I see problems in strategy, tactics and execution that have completely undermined user trust and loyalty.

Bookmarklets

The one mission critical feature which fuels the entire enterprise falls into disrepair. Seriously? This is unacceptable. The bookmarklets that allow users to bookmark and tag links were broken for long stretches of time and continue to be rickety and unreliable. This lack of support is akin to disrespect of Delicious users.

Stacks

Here’s how they work. Select some related links, plug them into a stack and watch the magic happen. You can customize your stack by choosing images to feature, and by adding a title, description and comment for each link. Then publish the stack to share it with the world. If you come across another stack you like, follow it to easily find it again and catch any updates.

Instead of the nearly frictionless interaction we’ve grown accustomed to, we’re now asked to perform additional and duplicative work. I’ve already created ‘stacks’ by bookmarking links with appropriate tags. Want to see a stack of links about SEO, look at my bookmarks that are tagged SEO. It doesn’t get much more simple than that.

Not only have they introduced complexity into a simple process, they’ve perverted the reason for bookmarking links. The beauty of Delicious was that you were ‘curating’ without trying. You simply saved links by tags and then one day you figured out that you had a deep reservoir of knowledge on a number of topics.

Stacks does the opposite and invites you to think about curation. I’d argue this creates substantial bias, invites spam and is more aligned with the dreck produced by Squidoo.

Here’s another sign that you’ve introduced unneeded complexity into a product.

Delicious Describes Stacks

In just one sentence they reference stacks, links, playlists and topics. They haven’t even mentioned tags! Am I creating stacks or playlists? If I’m a complete novice do I understand what ‘stack links’ even means?

Even if I do understand this, why do I want to do extra work that Delicious should be doing for me?

Design

Design over Substance

The visual makeover doesn’t add anything to the platform. Do pretty pictures and flashy interactions really help me discover content? Were Delicious users saying they would use the service more if only it looked prettier? I can’t believe that’s true. Delicious had the same UI for years and yet continued to be a popular service.

Delicious is a utilitarian product. It’s about saving, retrieving and finding information.

Sure, Flipboard is really cool but just because a current design pattern is in vogue doesn’t mean it should be applied to every site.

UX

There are a number of UX issues that bother me but I’ll highlight the three that have produced the most ire. The drop down is poorly aligned causing unnecessary frustration.

Delicious Dropdown Alignment

More than a few times I’ve gone across to to click on one of the drop down links only to have it disappear before I could finish the interaction.

The iconography is non-intuitive and doesn’t even have appropriate hover text to describe the action.

Delicious Gray Icons

Delicious Icons are Confusing

Does the + sign mean bookmark that link? What’s the arrow? Is that a pencil?

Now, I actually get the iconography. But that’s the problem! I’m an Internet savvy user, yet the new design seems targeted at a more mainstream user. Imagine if Pinterest didn’t have the word ‘repin’ next to their double thumbtack icon?

Finally, the current bookmarklet supports the tag complete function. You begin typing in a tag and you can simply select from a list of prior tags. This is a great timesaver. It even creates a handy space at the end so you can start your next tag. Or does it?

Delicious Tag Problems

WTF!? Why is my tag all muddled together?

Delicious improved tagging by allowing spaces in tags. That means that all tags have to be separated by commas. I get that. It’s not the worst idea either. But the tag complete feature should support this new structure. Because it looks like it functions correctly by inserting a space after the tag. I mean, am I supposed to use the tag complete feature and then actually backspace and add a comma?

It’s not the best idea to make your users feel stupid.

Uptime

Delicious Unavailable Page

The service has been unstable, lately as poor as it was at the height of Twitter’s fail whale problem. I’ve seen that empty loft way too much.

What Should Delicious Do Instead?

It’s easy to bitch but what could Delicious have done instead? Here’s what I think they should have (and still could) do.

Filtering

An easy first step to improve Delicious would be to provide a better way to filter bookmarks. The only real way to do so right now is by adding additional tags. It would have been easy to introduce time (date) and popularity (number of times bookmarked) facets.

They could have gone an extra step and offered the ability to group bookmarks by source. This would let me see the number of bookmarks I have by site by tag. How many times have I bookmarked a Search Engine Land article about SEO? Not only would this be interesting, it maps to how we think and remember. You’ll hear people say something like: “It was that piece on management I read on Harvard Business Review.”

There are a tremendous number of ways that the new team could have simply enhanced the current functionality to deliver added value to users.

Recommendations

Recommendation LOLcat

Delicious could create recommendations based on current bookmark behavior and tag interest. The data is there. It just needs to be unlocked.

It would be relatively straightforward to create a ‘people who bookmarked this also bookmarked’ feature. Even better if it only displayed those I haven’t already bookmarked. That’s content discovery.

This could be extended to natural browse by tag behavior. A list of popular bookmarks with that tag but not in my bookmarks would be pretty handy.

Delicious could also alert you when it saw a new bookmark from a popular tag within your bookmarks. This would give me a quick way to see what was ‘hot’ for topics I cared about.

Recommendations would put Delicious in competition with services like Summify, KnowAboutIt, XYDO and Percolate. It’s a crowded space but Delicious is sitting on a huge advantage with the massive amount of data at their disposal.

Automated Stacks

Instead of introducing unnecessary friction Delicious could create stacks algorthmically using tags. This could be personal (your own curated topics) or across the entire platform. Again, why Delicious is asking me to do something that they can and should do is a mystery to me.

Also, the argument that people could select from multiple tags to create more robust stacks doesn’t hold much water. Delicious knows which tags appear together most often and on what bookmarks. Automated stacks could pull from multiple tags.

The algorithm that creates these stacks would also constantly evolve. They would be dynamic and not prone to decay. New bookmarks would be added and bookmarks that weren’t useful (based on age, lack of clicks or additional bookmarks) would be dropped.

Delicious already solved the difficult human element of curation. It just never applied appropriate algorithms to harness that incredible asset.

Social Graph Data

Delicious could help order bookmarks and augment recommendations by adding social graph data. The easiest thing to do would be to determine the number of Likes, Tweets and +1s each bookmark received. This might simply mirror bookmark popularity though. So you would next look at who saved the bookmarks and map their social profiles to determine authority and influence. Now you could order bookmarks that were saved by thought leaders in any vertical.

A step further, Delicious could look at the comments on a bookmarked piece of content. This could be used as a signal in itself based on the number of comments, could be mined to determine sentiment or could provide another vector for social data.

Trunk.ly was closing in on this since they already aggregated links via social profiles. Give them your Twitter account and they collect and save what you Tweet. This frictionless mechanism had some drawbacks but it showed a lot of promise. Unfortunately Trunk.ly was recently purchased by Delicious. Maybe some of the promise will show up on Delicious but the philosophy behind stacks seems to be in direct conflict with how Trunk.ly functioned.

Analytics

Delicious could have provided analytics to individuals as to the number of times their bookmarks were viewed, clicked or re-bookmarked. The latter two metrics could also be used to construct an internal influence metric. If I bookmark something because I saw your bookmark, that’s essentially on par with a retweet.

For businesses, Delicious could aggregate all the bookmarks for that domain (or domains), providing statistics on the most bookmarked pieces as well as when they are viewed and clicked. A notification service when your content is bookmarked would also be low-hanging fruit.

Search

Delicious already has search and many use it extensively to find hidden gems from both the past and present. But search could be made far better. In the end Delicious could have made a play for being the largest and best curated search engine. I might be biased because of my interest in search but this just seems like a no-brainer.

Revenue

Building a PPC platform seems like a good fit if you decide to make search a primary feature of the site. It could even work (to a lesser extent) if you don’t feature search. Advertisers could pay per keyword search or tag search. I doubt this would disrupt user behavior since users are used to this design pattern thanks to Google.

Delicious could even implement something similar to StumbleUpon, allowing advertisers to buy ‘bookmark recommendations’. This type of targeted exposure would be highly valuable (to users and advertisers) and the number of bookmarks could provide long-term traffic and benefits. Success might be measured in a new bookmarks per impression metric.

TL;DR

The new Delicious is a step backward, abandoning simplicity and neglecting mechanisms that build replenishing value. Instead management has introduced complexity and friction while concentrating on cosmetics. The end result is far worse than the neglect Delicious suffered at the hands of Yahoo.

The Knuckleball Problem

December 08 2011 // Marketing + Rant + Web Design // 4 Comments

The knuckleball is a very effective pitch if you can throw it well. But not many do. Why am I talking about arcane baseball pitches? Because the Internet has a knuckleball problem.

Knuckleball

Image from The Complete Pitcher

The Knuckleball Problem

I define the knuckleball problem as something that can be highly effective but is also extremely difficult. The problem arises when people forget about the latter (difficulty) and focus solely on the former (potential positive outcome).

Individuals, teams and organizations embark on a knuckleball project with naive enthusiasm. They’re then baffled when it isn’t a rousing success. In baseball terms that means instead of freezing the hitter, chalking up strikeouts and producing wins you’re tossing the ball in the dirt, issuing walks and running up your ERA.

If a pitcher can’t throw the knuckleball effectively, they don’t throw the knuckleball. But in business, the refrain I hear is ‘X isn’t the problem, it’s how X was implemented‘.

This might be true, but the hidden meaning behind this turn of phrase is the idea that you should always attempt to throw a knuckleball. In reality you should probably figure out what two or three pitches you can throw to achieve success.

Difficulty and Success

The vast majority of pitchers do not throw the knuckleball because it’s tough to throw and produces a very low success rate. Most people ‘implement’ or ‘execute’ the pitch incorrectly. Instead pitchers find a mix of pitches that are less difficult and work to perfect them.

Yet online, a tremendous number of people try to throw knuckleballs. They’re trying something with a high level of difficulty instead of finding less difficult (perhaps less sexy or trendy) solutions. And there is a phalanx of consultants and bloggers who seem to encourage and cheer this self-destructive behavior.

Knuckleballs

In general I think mega menus suck. Of course there are exceptions but they are few and far between. The mega menu is a knuckleball. Sure you can attempt it, but the odds are you’re going to screw it up. And there are plenty of other ways you can implement navigation that will be as or even more successful.

When something has such a high level of difficulty you can’t just point to implementation and execution as the problem. When a UX pattern is widely misapplied is it really that good of a UX pattern?

Personas also seem to be all the rage right now. Done the right way personas can sometimes deliver insight and guidance to a marketing team. But all too often the personas are not rooted in real customer experiences and devolve into stereotypes that are then used as weapons in cross-functional arguments meetings. “I’m sorry, but I just don’t think this feature speaks to Concerned Carl.”

Of course implementation and execution matter. But when you consistently see people implementing and executing something incorrectly you have to wonder whether you should be recommending it in the first place.

Pitching coaches aren’t pushing the knuckleball on their pitching staffs.

Can You Throw a Knuckleball?

Cat Eats Toy Baseball Players

The problem is most people think they can throw the online equivalent of the knuckleball. And unlike the baseball diamond the feedback mechanism online is far from direct.

Personas are created and used to inform your marketing strategy and there is some initial enthusiasm and some minor changes but over time people get tired of hearing about these people and the whole thing peters out along with the high consulting fees which are also conveniently forgotten.

The hard truth is most people can’t throw the knuckleball. And that’s okay. You can still be a Cy Young Award winner. Tim Lincecum does not throw a knuckleball.

How (and When) To Throw The Knuckleball

This doesn’t mean you shouldn’t be taking risks or attempt to throw a knuckleball once in a while. Not at all.

However, you shouldn’t attempt the knuckler simply because it is difficult or ‘more elegant’ or the hottest new fad. You can take plenty of risks throwing the slider or curve or change up, all pitches which have a higher chance of success. In business terms the risk to reward ratio is far more attractive.

If you’re going to start a knuckleball project you need to be clear about whether you have a team that can pull it off. Do you really have a team of A players or do you have a few utility guys on the team?

Once you clear that bit of soul searching you need to be honest about measuring success. A certain amount of intellectual honesty is necessary so that you can turn to the team and say, you tossed that one in the dirt. Finally, you need a manager who’s willing to walk to the mound and tell the pitcher to stop futzing with the knuckleball and start throwing some heat.

TL;DR

The Internet has a knuckleball problem. Too many are attempting the difficult without understanding the high probability of failure while ignoring the less difficult that could lead to success.

Not Provided Keyword Not A Problem

November 21 2011 // Analytics + Rant + SEO // 15 Comments

Do I think Google’s policy around encrypting searches (except for paid clicks) for logged-in users is fair? No.

Fair Is Where You Get Cotton Candy

But whining about it seems unproductive, particularly since the impact of (not provided) isn’t catastrophic. That’s right, the sky is not falling. Here’s why.

(Not Provided) Keyword

By now I’m sure you’ve seen the Google Analytics line graph that shows the rise of (not provided) traffic.

Not Provided Keyword Google Analytics Graph

Sure enough, 17% of all organic Google traffic on this blog is now (not provided). That’s high in comparison to what I see among my client base but makes sense given the audience of this blog.

Like many others (not provided) is also my top keyword by a wide margin. I think seeing this scares people but it makes perfect sense. What other keyword is going to show up under every URL?

Instead of staring at that big aggregate number you have to look at the impact (not provided) is having on a URL by URL basis.

Landing Page by Keywords

To look at the impact of (not provided) for a specific URL you need to view your Google organic traffic by Landing Page. Then drill down on a specific URL and use Keyword as your secondary dimension. Here’s a sample landing page by keywords report for my bounce rate vs exit rate post.

Landing Page by Keyword Report with Not Provided

In this example, a full 39% of the traffic is (not provided). But a look at the remaining 61% makes it pretty clear what keywords bring traffic to this page. In fact, there are 68 total keywords in this time frame.

Keyword Clustering Example

Clustering these long-tail keywords can provide you with the added insight necessary to be confident in your optimization strategy.

(Not Provided) Keyword Distribution

The distribution of keywords outside of (not provided) gives us insight into the keyword composition of (not provided). In other words, the keywords we do see tell us about the keywords we don’t.

Do we really think that the keywords that make up (not provided) are going to be that different from the ones we do see? It’s highly improbable that a query like ‘moonraker steel teeth’ is driving traffic under (not provided) in my example above.

If you want to take things a step further you can apply the distribution of the clustered keywords against the pool of (not provided) traffic. First you reduce the denominator by subtracting the (not provided) traffic from the total. In this instance that’s 208 – 88 which is 120.

Even without any clustering you can take the first keyword (bounce rate vs. exit rate) and determine that it comprises 20% of the remaining traffic (24/120). You can then apply that 20% to the (not provided) traffic (88) and conclude that approximately 18 visits to (not provided) are comprised of that specific keyword.

Is this perfectly accurate? No. Is it good enough? Yes. Keyword clustering will further reduce the variance you might see by specific keyword.

Performance of (Not Provided) Keywords

The assumption I’m making here is that the keyword behavior of those logged-in to Google doesn’t differ dramatically from those who are not logged-in. I’m not saying there might not be some difference but I don’t see the difference being large enough to be material.

If you have an established URL with a history of getting a steady stream of traffic you can go back and compare the performance before and after (not provided) was introduced. I’ve done this a number of times (across client installations) and continue to find little to no difference when using the distribution method above.

Even without this analysis it comes down to whether you believe that query intent changes based on whether a person is logged-in or not? Given that many users probably don’t even know they’re logged-in, I’ll take no for 800 Alex.

What’s even more interesting is that this is information we didn’t have previously. If by chance all of your conversions only happen from those logged-in, how would you have made that determination prior to (not provided) being introduced? Yeah … you couldn’t.

While Google has made the keyword private they’ve actually broadcast usage information.

(Not Provided) Solutions

Keep Calm and SEO On

Don’t get me wrong. I’m not happy about the missing data, nor the double standard between paid and organic clicks. Google has a decent privacy model through their Ads Preferences Manager. They could adopt the same process here and allow users to opt-out instead of the blanket opt-in currently in place.

Barring that, I’d like to know how many keywords are included in the (not provided) traffic in a given time period. Even better would be a drill-down feature with traffic against a set of anonymized keywords.

Google Analytics Not Provided Keyword Drill Down

However, I’m not counting on these things coming to fruition so it’s my job to figure out how to do keyword research and optimization given the new normal. As I’ve shown, you can continue to use Google Analytics, particularly if you cluster keywords appropriately.

Of course you should be using other tools to determine user syntax, identify keyword modifiers and define query intent. When keyword performance is truly in doubt you can even resort to running a quick AdWords campaign. While this might irk you and elicit tin foil hat theories you should probably be doing a bit of this anyway.

TL;DR

Google’s (not provided) policy might not be fair but is far from the end of the world. Whining about (not provided) isn’t going to change anything. Figuring out how to overcome this obstacle is your job and how you’ll distance yourself from the competition.

Worst SEO Title Ever

September 20 2011 // Rant + SEO // 16 Comments

Do as I say, not as I do. That seems to be Google’s philosophy when it comes to blog optimization.

Worst SEO Title Ever

Worst SEO Title Ever

What has finally pushed me over the edge into rant mode? It’s today’s Google+ announcement.

Bad Google+ Blog Post Title

A bunch of numbers for your title. Really? Instead maybe you’d, you know, want to mention the introduction of search or that Google+ was now open to everyone. Those are actually really interesting and noteworthy items.

This isn’t a John Barth novel. The meta information around the number of improvements isn’t really relevant. Really, it’s not.

What query intent are you trying to match here? And yes, that matters.

Snippet Optimization

Google also continues to fail on snippet optimization. Yes, we know that the meta description isn’t a ranking factor. But the description is more important today since it’s used in the transmission of information to other platforms. So what does the snippet for this post look like?

Bad Google+ Snippet

At a glance can you tell what this is about? I certainly can’t. The default image here is useless, the title is nonsense and the description simply tells me that it’s available in other languages. Google can count to 100, seemingly in different languages. Congratulations.

Best Practices and Role Models

Does everyone have to follow best practices? No. All of this is optional. But Google is in a position where they should be setting an example. Google might want to take the Charles Barkley approach, but like it or not, you are a role model.

Or perhaps this is a deliberate thumb in the eye to the SEO community? We know that Google is willing to change titles when they think they’re not quite right. So maybe they just don’t think any of this is necessary? But I doubt that’s the case. Remember the adage about malice.

So please Google, take the time to perform the minimum of optimization on your vast collection of blogs (or give me and my team a call and we’ll get you square.) It’s good for you and it’s good for the search community.

[Update] Well, it looks like the Google Mobile Blog wants to fight for the Worst SEO Title Ever crown with their own numbers post.

 

PageRank Ponzi

September 09 2011 // Rant + SEO // 12 Comments

Why are you still submitting your site and articles to directories? Sure, there was a time when directories were valuable. But that time has passed. So stop feeding their business and build your own instead.

Totally Flabbergasted LOLcat

Page Rank Ponzi

Directories are essentially a form of PageRank ponzi. They use your content to build their business – to build their trust and authority – and, in exchange, lease a small fraction of that trust and authority (e.g. PageRank) back to you.

You either give away or actually pay to provide them with content. They take your assets, gladly, and use it to do what you should be doing. Even if you get a small benefit from this exchange, you’re getting the short end of the stick.

Directory Heyday

There was a time when directories were useful and valuable. From the mid-to-late 90s to around 2003, directories were used by many to find sites and content. This was before tabbed browsing and broadband connections made it easy to get from one site to another. This was before search became the dominant way to navigate the web. This was before social platforms allowed you to tap your social graph and crowdsource information.

One only needs to look at the search volume for the term ‘web directory’ to see that this is an outdated method of online discovery.

Search Trend for Web Directory Searches

Distribution

In the directory heyday it may have been difficult to get your site, article or blog post distributed. The web was not nearly as connected or fluid.

But today we have blogging platforms, a robust social graph and numerous social media outlets that give you an opportunity to capitalize on your own intellectual property instead of giving it away to others for peanuts.

We Are The Directory

Whether you call it curation or crowdsourcing there are other repositories that mimic and exceed the traditional directory. You might search Delicious. In fact, more people should. Or you might try out Trunk.ly.

We’re doing the work of directories every day.

Caffeine

In June of 2010, Google launched Caffeine and increased their ability to crawl and index the web. This was one of the last pieces of the puzzle in making directories obsolete.

Previously, directories might have been able to quickly surface new sites or content that hadn’t yet been found by Google. But that’s just not the case today. Google finds new content even in the dark and dusty corners of the Internet where Geocities pages lurk and survive.

Google Directory

So what does Google think about directories today?

Google Directory No Longer Available Message

Google shut down their directory. Read that again and think about what it means for the future and value of directories. And don’t get me started on the utter collapse of DMOZ. (No, I’m not even going to link there.)

As an aside, Google may want to consider a folder level URL removal so directory results (which return a 404) don’t clutter up SERPs.

Directory Spam

Most web directories are hastily thrown together arbitrage sites that serve as outposts for spam. Here’s a excerpt from an email sent to me by an ‘SEO Consultant’.

Directory Spam

This is not SEO, at least not the SEO I practice. Some may reject this carpet bombing approach but subscribe to the idea that a handful of paid directories are worthwhile.

I say save your money.

Paid Link or Paid Listing?

Jack McCoy from Law & Order

Frankly, I’m still a bit irked that Google doesn’t view a paid listing as a paid link. The argument for paid directories is that they provide a certain level of curation that makes them valuable. You’re paying for someone to curate that directory – not for the link. This seems a very thin argument at best, and a bunch of claptrap at worst. Most, if not all, directories are pretty much a free-for-all as long as what you’re submitting isn’t complete spam or off topic. The level of curation is marginal, and I’m being nice.

Not only that, but it comes down to intent. For some reason I hear Jack McCoy yelling ‘intent follows the bullet’. It’s not a perfect analogy, but the general idea is that intent matters. Today, the intent for a directory listing is, quite simply, to secure a back link. So, what exactly is the difference between a paid link and a paid listing? There is none as far as I can tell.

Link Value

REM Out of Time Cover Art

How valuable is that directory link anyway? I’m telling you that the value of these links declines every day. People aren’t using these sites. Newer technologies have replaced directories in the information ecosystem. The closure of the Google Directory should be a wake up call to anyone still clinging to this practice.

TL;DR

Traditional directories are an obsolete method of information discovery. Even if they provide some small benefit today, you’re paying a hefty price to support someone else’s dying business model. Stop PageRank ponzi and invest in the future and yourself instead.

Comment Censorship

August 07 2011 // Rant + Social Media + Technology // 15 Comments

In the past month I’ve left a number of comments on blogs only to find they were never published.

Fry Meme Spam or Blog Censorship

I’d like to believe that the blog owners simply didn’t see my comment. That it fell into their spam queue which they rarely, if ever, look at. Because the alternative is that they saw the post and decided to suppress it. Now, it’s their blog – their little corner of the Internet – but this type of censorship is troubling.

Comments Content

What about the content of my comments? To be fair, in some instances I was disagreeing with some or all of the content in that post. But I did so in a constructive manner, using links to my own thoughts on the topic or to other material to help round out my argument.

I regularly publish comments on this blog that are contrary to my own opinion. One only has to look at the comments on my Stop Writing For People post for examples. I’m free to respond and defend myself, but having the debate in the open is important. It builds trust, much like having bad reviews on a product is actually a good thing.

Comments are incredibly valuable because they provide additional information on the content. They make your content better through clarification, confirmation, addition and debate.

Comments = Content.

Comments are a rich source of meta information that deliver value to both readers and search engines. This extends to links as well! Relevant links in comments help create a web of information that users now and in the future will find useful.

Yet it is those links that may be at the root of the problem.

Comment Spam

It’s like the Internet version of a plague of locusts. One of the most popular ways to combat comment spam is to screen comments that have links. This is one of the default setting in Akismet.

It makes sense since many spammers will drop a link or links in comments. But links are not the problem. Spammers are the problem.

What’s wrong with contextual links to relevant content? This is not behavior that should be penalized. In fact, it should be encouraged. In many ways, the comment spam problem threatens the link graph.

ratio of comment spam to real comments

Not only that but, anecdotally, it seems that comment spam sometimes pushes people to disable comments altogether. When the ratio of comment spam to real comments is too high, many simply give up. I understand the decision but it’s depressing that it gets to that point.

Outsourcing

Fed up with comment spam and general comment management, have we decided to outsource engagement to social networks? Twitter, Facebook, LinkedIn, and Google+ are all happy to provide venues in which comments can flourish. Make no mistake, these venues understand the value of comments.

Is our obsession with amplification and generating social proof robbing us of the real value of comments and conversation? Certainly there is some hope that it’s like a rubber band. The content goes out, but then snaps back, drawing more comments to your content. It works to a certain extent, but by how much and at what cost is an interesting debate.

The Filter Bubble

Of course these bloggers may have seen my comment and simply decided not to publish it. Eli Pariser argues that personalization and ‘invisible algorithmic editing’ as a real danger but I think comment censorship (whether intentional or accidental) is the true menace.

I believe much of the hype around the filter bubble is FUD. Personalization is rather minimal in most cases though I do agree with Gabriel Weinberg’s view of how to deal with personalization.

Personalization is not a black and white feature. It doesn’t have to be on or off. It isn’t even one-dimensional. At a minimum users should know which factors are being used and at best they should be able to choose which factors are being used, to what degree and in what contexts.

Personalization deals with the fact that some content isn’t being made readily visible. Comment censorship excises content from the Internet altogether.

Identity

So what could help get us out of this morass? How can we ensure comments are once again a vital part of the content ecosystem? Identity.

Identity

The reason why many embraced Facebook comments was because comments are attached to an identity. Not only that, but an identity that people cared about. This obviates the need for aggressive moderation. You might run into a troll, but it’ll be a troll you can clearly identify and block.

Identity essentially stops comment spam because you can’t post as Best Miami Attorneys. Comment moderation is suddenly manageable again.

Censorship

A commenting system that uses identity removes most of the uncertainty around comment censorship. If my comment isn’t published, it’s likely because that blogger made an active decision to toss it into the HTML version of The Bermuda Triangle.

Cat Censors Blog Comments

If the filter bubble can be managed through making personalization transparent, so too can comment censorship. A third-party, identity-backed comment system could track the number of comments censored on each blog. A grade or score could then be shown to let users know how much of the conversation was being censored. In some ways it would be like Charity Navigator but for blogs.

So perhaps the blogger who touts the benefits of community actually censors 32% of blog comments. That might be an interesting thing to know.

Could this get messy? Sure. But you can build a system of checks and balances.

Reputation

Bad Reputation by Joan Jett

Joan Jett might not care about her bad reputation but you should. Whether it’s a thumbs-up, thumbs-down, number of Likes, sentiment analysis, length of comments, spelling and grammar or other metrics, a savvy comment system could begin to assign reputation to each user.

So the censorship percentage wouldn’t be flat in nature. If you blocked a known troll, no worries. If you censored someone who had a history of abusive comments full of foul language, no problem.

On the other hand, it would be disturbing if you censor someone who consistently adds value to conversations. The reputation of those you censor would matter.

Confidence

I’d like to be confident that I’m not missing good comments that wind up going into spam.

I’d like to be confident that if I take the time and effort to comment on a blog that it will be published and, hopefully, spark further comment and conversation.

I’d like to be confident that the comments I read are not biased and simply a form of self-curated cheerleading.

“Confidence is contagious. So is lack of confidence.” – Vince Lombardi

The Internet desperately needs more confidence.

Yahoo Email Hacked

May 23 2011 // Rant + Technology // 419 Comments

(IMPORTANT: Before I get to my story, if your Yahoo! email has been hacked I recommend that you immediately change your password, update your security questions and ensure your Yahoo! Mobile and Y! Messenger are both up-to-date. You should also visit Yahoo! Email Abuse Help and use this process if you are unable to login to your Yahoo! account. Also, make sure to read the comments on this post since there is a tremendous amount of good information there as well.)

(UPDATE 12/13/11: Yahoo has introduced second sign-in verification as an added security measure. It will require that you add a mobile phone number and verify it via a text message. Here’s the direct link to start using second sign-in verification.)

It happened just before we arrived at the San Francisco Zoo. We are at a red light on Sloat Boulevard when my phone started to vibrate.

Buzz. Buzz. Buzz. Buzz. Buzz. Buzz. Buzz. Buzz. Buzz. Buzz. Buzz. Buzz. Buzz.

Had the rapture come a day late? No. I was getting undeliverable messages. Lots of them. My Yahoo email had been hacked!

admiral akbar star wars its a trap spoof

Here are the two important lessons I learned as a result.

I Have Good Friends

I didn’t want our day at the Zoo ruined, me staring into my phone resetting passwords and figuring out what happened. So I put the problem on the back burner and proceeded to have a fun family day.

But I did take time to quickly tap out a response to people who replied to the spam coming from my hijacked account. Why? Because they took the time and effort to give me a heads up that I had a problem. These were good people. Good friends.

The thing is, I’d gotten a number of these same emails lately from other hacked Yahoo accounts. I figured these people knew they’d been compromised and I didn’t need to respond. With the shoe on the other foot, I realized those emails were comforting even though I was well aware of the problem.

I’ll shoot off an email the next time I get a hacked email from someone.

Yahoo Email Security Failed

The odds are that I will get another one of those emails because I learned just how easy Yahoo makes it for hackers.

Upon getting home I went about securing my account. On a lark, I checked Yahoo’s ‘View your recent login activity’ link.

yahoo recent login activity

Sure enough at 10:03 AM my account was accessed from Romania. This obvious login anomaly didn’t set off any alarms? Shouldn’t my security questions have been presented in this scenario? I have never logged in from Romania before.

I’ve never logged in from outside the US. Yahoo knows this. In fact, Yahoo knows quite a bit about my location.

yahoo location history

My locations puts me in three states: California, New York and Pennsylvania. I also have location history turned on, so it’s not just my own manually saved locations (some of which are ancient), but Yahoo’s automated location technology keeping track of me.

Do you see Romania in this list? I don’t.

Why is Yahoo making it this easy for spammers to hijack accounts? Make them work a little bit! At a minimum, make them spoof their location.

Yahoo should have noted this anomaly and used my security questions to validate identity. I still would have had to change my password (which wasn’t that bad) but I would have avoided those embarrassing emails.

A simple rule set could have been applied here where users are asked to validate identity if the login (even a successful one) is outside of a 500 mile radius of any prior location.

I’ve had a Yahoo account for over 10 years without a problem, even as I moved my business accounts over to Gmail.

Yesterday I thanked those friends who had my back. Unfortunately, Yahoo wasn’t one of them.

WordPress Duplicate Content

April 27 2011 // Rant + SEO + Technology // 21 Comments

In February Aaron Bradley sent me an email to let me know that I had a duplicate content problem on this blog. He had just uncovered and rectified this issue on his own blog and was kind enough to give me a heads up.

Comment Pagination

The problem comes in the way that WordPress handles comment pagination. The default setting essentially creates a duplicate comment page.

Here’s what it looks like in the wild. Two pages with the same exact content.

http://blog.wolframalpha.com/2011/04/18/new-age-pyramids-enhance-population-data/comment-page-1/

http://blog.wolframalpha.com/2011/04/18/new-age-pyramids-enhance-population-data

That’s not good. Not good at all.

Comment-Page-1 Problem

The comment-page-1 issue offends my own SEO sensibilities, but how big of a problem is it really?

WordPress Spams Google

There are 28 million inurl results for comment-page-1. 28 million!

Do the same inurl search for comment-page-2 and you get about 5 million results. This means that only 5 million of these posts attracted enough comments to create a second paginated comment page. Subtract one from the other and you wind up with 23 million duplicate pages.

The Internet is a huge place so this is probably not a large percentage of total pages but … it’s material in my opinion.

Change Your Discussion Settings

If you’re running a WordPress blog I implore you to do the following.

Go to your WordPress Dashboard and select Settings –> Discussions.

How To Fix Comment-Page-1 Problem

If you regularly get a lot of comments (more than 50 in this default scenario) you might want to investigate SEO friendly commenting systems like Disqus, IntenseDebate or LiveFyre.

Unchecking the ‘break comments into pages’ setting will ensure you’re not creating duplicate comment pages moving forward. Prior comment-page-1 URLs did redirect, but seemed to be doing so using a 302 (yuck). Not satisfied I sought out a more permanent solution.

Implement an .htaccess RewriteRule

It turns out that this has been a known issue for some time and there’s a nice solution to the comment-page-1 problem in the WordPress Forum courtesy of Douglas Karr. Simply add the following rewrite rule to your .htaccess file.

RewriteRule ^(.*)/comment-page-1/ $1/ [R=301,L]

This puts 301s in place for any comment-page-1 URL. You could probably use this and keep the ‘break comments into pages’ setting on, which would remove duplicate comment-page-1 URLs but preserve comment-page-2 and above.

Personally, I’d rather have the comments all on one page or move to a commenting platform. So I turned the ‘break comments into pages’ setting off and went a step further in my rewrite rule.

RewriteRule ^.*/comment-page-.* $1/ [R=301,L]

This puts 301s in place for any comment-page-#. Better safe than sorry.

Don’t Rely on rel=canonical

Many of the comment-page-1 URLs have a rel=canonical in place. However, sometimes it is set up improperly.

Improper Rel=Canonical

Here the rel=canonical actually reinforces the duplicate comment-page-1 URL. I’m not sure if this is a problem with the Meta SEO Pack or simple user error in using that plugin.

Many times the rel=canonical is set up just fine.

Canonical URL from All-In-One SEO Pack

The All in One SEO Pack does have a Canonical URL option. I don’t use that option but I’m guessing it probably addresses this issue. The problem is that rel=canonical doesn’t stick nearly as well as a 301.

Comment-Page-1 in SERP

So even though this post from over three months ago has a rel=canonical, the comment-page-1 URL is still being returned. In fact, there are approximately 110 instances of this on this domain alone.

Comment Page 1 Site Results

Stop Comment-Page-1 Spam

23 million pages and counting. Sure, it would be nice if WordPress would fix this issue, but short of that it’s up to us to stop this. Fix your own blog and tell a friend.

Friends don’t let friends publish duplicate content.