Blind Five Year Old

The other day I received an interesting email from Yahoo!

It’s pretty easy to read between the lines here. In fact, little line reading is necessary. The new comScore beacon is providing more accurate results. Yahoo is not currently participating in the beacon program. Yahoo wasn’t keen on the “apples-to-oranges” comparison that “could create confusion for advertisers” because it would likely negatively impact their display business.

Don’t Forget Yahoo!

Bashing Yahoo! seems to be the cool thing to do these days, and they’ve certainly driven themselves into a ditch. But Yahoo! still holds a powerful position as a portal, content and email provider. This email seems like a not-so-gentle reminder that Yahoo! is still a 900 pound gorilla in some circles.

Beacons and Panel Data

The other takeaway here is the fact that beacons are fast becoming the best way to measure traffic. I see comScore’s introduction of beacon technology as a direct reaction to Quantcast.

The rise of Google Analytics allows more and more companies to know exactly how much traffic they receive. The result of this knowledge is a growing dissatisfaction with panel based measurements that aren’t just inaccurate but are sometimes flat out wrong.

No More Hand Waving

Whether it was the Web 1.0 darling Alexa or recent upstart Compete, panel based services continue to fail. The difference this time around is that we have beacons (like Google Analytics and Quantcast) that let us know when they fail and by how much.

So while Yahoo! has secured a 6-month repreive, the future will be in accurate and transparent traffic measurement.

This morning Microsoft announced a 10 year search partnership with Yahoo!

In simple terms, Microsoft will now power Yahoo! search while Yahoo! will become the exclusive worldwide relationship sales force for both companies’ premium search advertisers.

But it’s not all simple and straight-forward. If Bing now powers Yahoo! search, what happens to Yahoo! Paid Inclusion?

What is Paid Inclusion?

Yahoo! Paid Inclusion is a pay-per-click product that allows an advertiser to pay for a listing in the search index. That’s right, you can pay to look like an ‘organic’ listing. Paid Inclusion is the dirty little secret that nobody at Yahoo! talks about and is only whispered about among advertisers.

Will Yahoo! Paid Inclusion Survive?

At first glance it might seem that Paid Inclusion will go the way of the dodo bird. Or will it? The second key term provides an opening.

Microsoft will acquire an exclusive 10 year license to Yahoo!’s core search technologies, and Microsoft will have the ability to integrate Yahoo! search technologies into its existing web search platforms;

While this statement is clearly about the Yahoo! search algorithm and (perhaps) ancillary products like SearchMonkey, it could also apply to Paid Inclusion.

The reach of Yahoo’s Paid Inclusion product has never been clear and, therefore, the actual revenue Paid Inclusion generates is also a bit of a mystery. Is Paid Inclusion revenue big enough to preserve through this partnership or not?

Paid Inclusion and SEO

Paid Inclusion already obscures Yahoo! SEO. If Paid Inclusion goes away, many advertisers who were getting traffic ahead of natural listings will suddenly be at the mercy of natural SERP and SEO. There could be a substantial traffic impact for these advertisers who have come to rely on a certain amount of traffic at a predictable ROI.

Should Paid Inclusion not survive the partnership, SEO – Bing SEO specifically – will become more important. If Paid Inclusion does survive, does it do so just on Yahoo! properties or would it be integrated into Bing results as well? If it is the latter, Paid Inclusion becomes much bigger.

In the final analysis, I don’t think Paid Inclusion revenue is large enough, can’t see Bing integrating Paid Inclusion and similarly can’t see Microsoft wanting to explain why searches on Yahoo! and Bing are materially different.

For these reasons I’m guessing that Paid Inclusion will be terminated.

UPDATE (July 30, 2009)

Danny Sullivan was able to land a question during the Yahoo! press conference.

I asked what happens to other things search like at Yahoo? What powered Yahoo News? What happens to the Yahoo Directory? Is Delicious search? And what happens to Yahoo paid inclusion?

Bartz: We have full flexibility on what to do within our own sites. Paid inclusion, we’ll decide on that later.

So Paid Inclusion still has a chance of surviving but it seems like it’s on life support.

Lately, you can’t go a day without Yahoo talking about their new open initiatives. Whether it’s Yahoo BOSS (Build your Own Search Service), Yahoo Pipes or Yahoo SearchMonkey, it’s all about openness and collaboration.

Yahoo SearchMonkey

Yahoo SearchMonkey is the most interesting initiative for search. During SMX West every Yahoo presenter seemed to have a SearchMonkey slide in their deck. I can understand why.

Using SearchMonkey, developers and site owners can use structured data to make Yahoo! Search results more useful and visually appealing, and drive more relevant traffic to their sites.

SearchMonkey turns dreary search results into something more interesting.

There are only a handful of default SearchMonkey apps currently running on Yahoo and the number of user apps is still a middling 100 or so. Yet, the idea seems right.

SearchMonkey is add-ons for search

That’s right. SearchMonkey apps are the equivalent of Firefox add-ons.

Add-ons extend Firefox, letting you personalize your browsing experience. Take a look around and make Firefox your own.

Yahoo isn’t going to win on their search algorithm. While the result sets are subjective, Google is the perceived leader and perception is reality. In addition, Google is far more focused on perfecting search and iterating the algorithm.

That’s why SearchMonkey is a smart move. Taking a page from Firefox and WordPress, Yahoo is hoping that developers will make their results more appealing and usable through easy customization.

Firehoo

That’s not where the similarities stop. Yahoo has a similar position in search as Firefox does in browsers. They’re both up against large market share giants, Google and Internet Explorer respectively. Hence, they both fight against user inertia.

There are differences. Yahoo had the market share lead and let it slip from their grasp and many (myself included) would argue that Firefox is a better product (add-ons or not) then Internet Explorer.

Could Yahoo become the Firefox of search?

Maybe, but only if they focus and promote SearchMonkey apps to sites, developers and, most importantly, to users. Unfortunately Yahoo isn’t doing this and remains distracted and unfocused.

It would make search far more interesting if Yahoo got serious about SearchMonkey and fully implementing a customized search strategy to woo users back to Yahoo.

The future of search is numbered. No, it’s not what you think. I’m not predicting the demise of search. Quite the opposite.

Instead I’m talking about two trends that could have interesting implications on how both SEO and PPC campaigns are constructed.

The number of words per query is going up

Bill Tancer, General Manager of Global Research at Hitwise, shared the following statistics during a SMX West presentation.

The trend is clear. People are using more words in their search queries. The reasons behind the increase in words is debatable. Is it an increased comfort and savvy with search or frustration with the search results? Either way, long tail searches are on the rise and will likely trend this way for some time.

People are using numbers to reformulate queries

In a post titled Study on the Structure of Search Queries, Bill Slawski of SEO by the Sea discusses a Yahoo! research paper titled The Linguistic Structure of English Web-Search Queries (pdf). One of the findings was that people were using numbers to modify their searches.

The type of word most likely to be reformulated is “number.” Examples included changing a year (”most popular baby names 2007″ ! “most popular baby names 2008″), while others included model, version and edition numbers (”harry potter 6″ ! “harry potter 7″) most likely indicating that the user is looking at variants on a theme, or correcting their search need.

The data set for this research is from 2006. Yet, combined with the increase in words per query I’d theorize that numbers remain a powerful way to search today.

I’m the first person to warn against using yourself as an example but … I’m going to break that rule right now.

I often find myself using numbers, particularly years, when searching. If I’m doing research on a volatile topic (like SEO) I might come across ancient results from 2004. These are rarely helpful.

So I’ll begin to iterate and modify my query with years to find more relevant results. I had to do something similar when trying to cobble together Google’s share of search from October 2004 to October 2008.

Speaking of Google …

Google displays the date as a meta description prefix

Sometime last year Google began to insert a date prefix before the meta description in search results. They did this for pages in which Googlebot found a date – mostly blog posts.

I certainly noticed since some of my well crafted 150 character meta descriptions were suddenly being cut off because of the inserted date.

Why exactly would Google do this? They’ve certainly made it clear they like “fresh” content but they could present fresh content without the date. I can only surmise that Google believes users are looking for and will benefit from seeing the date.

Oddly, the date prefix does not seem to be searchable. When I search for ‘Magical Thinking by Augusten Burroughs 2007’ the same result is displayed but the visual treatment changes.

Once again Google tromps all over my well crafted meta description, but without the meta description date prefix. This seems to prove that the meta description date prefix isn’t searchable. But perhaps it should be.

Are you including numbers in your SEO and PPC strategies?

We know that users are using more words per query and that they’re fond of reformulating queries using numbers. This should be enough evidence to implement a robust modifier strategy, if you haven’t already.

Those who already have long tail programs should think about increasing the use of numbers as valid modifiers. As an example, wouldn’t review sites benefit from bidding on terms like ‘digital camera reviews 2009’ or ‘trek bicycle ratings 2008’?

Search (whether paid or organic) is about matching your content (or ad) to the user queries. Don’t let your programs become dated.

Yahoo is serving ads at the top of email alerts.

Most of my alerts are with Google, but a few of my legacy alerts still reside at Yahoo. So I’m not sure exactly when this began but it seems like it started as of December 1st.

Both search giants are looking for ways to sustain growth in a maturing market. Sponsored search and network search just aren’t going to move the needle in eyebrow arching ways anymore – particularly in a weak economic environment.

This is an interesting move by Yahoo in light of Google’s advertising extensions via Feedburner (FeedSense) and Google Friend Connect (FriendSense).

Remember, Yahoo still dominates news by a nearly 3 to 1 ratio. It’s nothing to sneeze at.

If we use traffic as a proxy for number of alerts it’s easy to see how Yahoo could quickly increase its reach. Effectiveness, well that’s another issue.

But with this move by Yahoo, can AlertSense be far behind?

Now is the time when bloggers go on the record with their thoughts for the year ahead. Place your bets! Stake your claim! Here’s mine.

Facebook Becomes A Portal

Realizing that social media and advertising is like oil and water, Facebook repositions itself as a portal leveraging Microsoft’s Live Search as the revenue model. This also might result in the potential acquisition of Netvibes to provide a more robust offering to compete with Yahoo!

Identity Systems Fail

Confused about the difference between OpenID, Facebook Connect and Google Friend Connect, users throw up their hands and decide not to use any of the above.

Video Advertising Succeeds

The adoption of video is surging faster than many expected. Longer formats and better quality will bring even more eyeballs who will grudgingly accept advertising.

Microformats Go Mainstream

Why they aren’t already is shocking. Nevertheless, in 2009 we’ll see microformats become a standard and search results will become far more robust as a result.

Banner CTR Becomes Obsolete

Brands will finally realize that measuring success by click through rate (CTR) isn’t working. Measurement ‘beyond clicks’ will be the new yardstick, whether that’s through new brand advertising measurement services like Vizu or through monitoring services like Brandwatch and Trackur among a gaggle of others.

RSS Adoption Spikes

Someone will (finally) figure out how to market RSS to ‘the masses’ who will grasp the sublime benefits of having content come to you instead of the other way around.

Kindle 2.0 Flops

Amid a weak economy Amazon releases the newest version of Kindle. Other readers have gained ground where Kindle has not and at the core Kindle is a solution without a problem.

Google Search Share Stalls

The move by Facebook (see above) causes a radical change in the search landscape. Microsoft passes Yahoo! for second place and talks about a Microsoft Yahoo merger are (unbearably) reignited.

FriendFeed Surpasses Twitter

FriendFeed adoption increases at an accelerated pace due to quick innovation, uncluttered design and an interface that lends itself to communication.

Someone ‘Dies’

Users reach social media overload and VCs get even more nervous about revenue creating social media shrinkage. In this instance ‘Dying’ means a company goes under or is purchased for a song. My short list includes Plurk, Twitter, Digg and Seesmic. This isn’t a reflection of the people or product but the inability to truly reach the mainstream with a service that has a profit model.

There are plenty of other things that I believe will happen in 2009, but they seem more obvious or an extension of current trends. Instead I tried to be a bit more bold, at least on a few of my predictions.

We’ll check in this time next year to see how I fared. In the meantime, feel free to comment and provide your feedback and reaction to my predictions.

Search pogosticking is defined as going back and forth from a search engine results page (SERP) to individual search result destination sites. The behavior may indicate poor search results since the user hasn’t been satisfied by one or more of the SERP results.

Google and Yahoo are clearly using, or thinking about using, this metric as part of their search algorithm. It makes a lot of sense and could provide an important user-defined input for the algorithm as well as guard against potential over-optimization. As an aside, access to this human feedback mechanism may be one of the reasons why Google hasn’t been eager to pay a premium for Digg.

How would search pogosticking influence SEO?

A user is presented with search results based on a specific query. The engine captures what result you click on and whether you return to that SERP and click on subsequent results and/or refine your query. (They could even conceivably determine the time between each click as a proxy for satisfaction with that result. This would reduce the chances of penalizing results that did deliver value.) The information can be aggregated for each query and compared to average pogosticking behavior by SERP rank.

So, let’s say that for a specific query the engine sees that the current SERP has an abnormally high pogostick rate for the top ranked result. This information could then be fed back into the algorithm as a negative signal, thereby reducing its SERP rank in a future algorithm update. Obviously, it would have to be statistically above the average pogostick rate for that position to be flagged. However, I’m certain both Google and Yahoo have smart folks who can calculate when it reaches significance.

Does search pogosticking sound far fetched?

It shouldn’t. Do a Google search and look at the page source. You’ll see something like the following next to each search result.

onmousedown="return rwt(this,'','','res','1'

The ‘res’ likely stands for result and the next parameter is the actual rank for that result on that page. More information on this rewrite and tracking behavior can be found at searchlores and blogmal.

Even without this technical knowledge it should be obvious they’re doing something like this if they’re providing users with customized search results. I’m not sure the ability to provide custom or personalized results is the true aim, but makes the collection of this information more palatable for many users.

SEO by the Sea recently posted about this topic including a relevant pogosticking patent application by Yahoo. I recommend you read his post as well.

How can I track pogosticking?

You can’t know for sure whether a user is pogosticking. Bounce rate can sometimes be a good indication, but there are instances where query intent is satisfied and a high bounce rate is the expected result. Many Q&A sites meet this criteria.

The best course of action is to review pages with high bounce rates and make certain you’re matching query intent and delivering value to those users. Post-click SEO is going to become a larger part of the equation, bluring the line between SEO and traditional design and UI.

Yahoo’s organic search results aren’t always organic. I don’t think the average Yahoo user has any idea that many Yahoo search results are paid listings via Yahoo’s paid inclusion product, Search Submit Pro. It’s sort of like buying organic apples and finding out some of them were sprayed with pesticide.

Yet, I can see my way past the ethical minefield paid inclusion presents. There is no expressed contract that states that the organic search results are 100% organic. That’s an assumption that many users make and which Yahoo simply fails to correct. There’s no 100% organic ‘sticker’ on the Yahoo search results.

The real question for me is how paid inclusion impacts search engine optimization (SEO). “Why do I get only 5% of my organic traffic from Yahoo when they’re 20% of the search market?” This is a common refrain I hear from a rising tide of companies. There are a number of possible explanations for this discrepancy.

Sites optimize for Google and not Yahoo

Google is the force in search with a 63% market share in August 2008. For that reason, sites optimize for Google first and Yahoo a distant second. The algorithm for each is different with Google placing far more weight on off page factors like links.

In particular Google seems to have a better quality link algorithm that helps place sites in appropriate neighborhoods of knowledge. Yahoo certainly looks at links but it seems tied to link popularity and link ratio. This means you have to overwhelm Yahoo with inbound links to really have an impact.

Result: Google optimization might not help you with Yahoo.

Yahoo’s index is slow to update

There’s no question that the Yahoo index is slow to update and is less agile than Google’s index. What Google indexes and categorizes in hours could take Yahoo weeks or months. It’s not that it doesn’t get into Yahoo’s index, it’s that Yahoo doesn’t categorize it so that it can be served to users for relevant searches. In many ways it might as well not exist.

Google is also constantly tweaking the search algorithm. If you’re working in SEO you’re accustomed to the changing nature of the Google algorithm. However, It’s unclear how often Yahoo changes its algorithm, but it’s far less than Google and it’s not nearly as big a deal when they do. The Yahoo search index has calcified.

Result: It could take a long time for your efforts to pay off on Yahoo.

Paid Inclusion takes up search engine result shelf space

While there are no guarantees that paid inclusion will get you top results, it certainly takes up shelf space on search results. As more companies use paid inclusion, the number of organic search results goes down, making it more and more difficult to optimize vital keywords.

It seems logical that Yahoo, particularly in its current financial situation, would want to increase the number of paid inclusion results. As a cost per click product it would mean a greater probability of generating revenue with every search engine result page.

What’s the keyword coverage for paid inclusion? What’s the average number of paid inclusion listings to organic listings? These paid inclusion statistics aren’t available, so it’s difficult to understand the real effect of paid inclusion. Instead we can only deduce from watching our SEO efforts and tracking organic search volume.

Result: Paid inclusion might not be the culprit, but it introduces another variable that defies tracking, thus obscuring all other SEO efforts.