Blind Five Year Old

This morning (thanks to a tip from Search Engine Roundtable) I began researching what looked like authorship snippets on Bing. While it’s only been an hour or so here’s what I’ve seen and what I think I’ve figured out.

People Snippets

The new faces (for the most part) showing up in Bing search results are not authorship snippets per se but are people snippets derived from entities. It’s about who the content is about rather than who created the content.

If you haven’t seen them already here’s what one looks like when you search for Lauren Cohan.

They look remarkably like the authorship snippets that Google has implemented but they’re most certainly different in their application.

Structured Data?

The first assumption here is that Bing might be using structured data to present these new snippets. Perhaps they’re using the person attribute in schema.org markup?

Not so much. There’s no structured data on this page and I’ve found plenty of others getting the people snippet that are devoid of mark-up. So if Bing isn’t using structured data, what are they using to match and identify people?

People Pages

Clearly they rely heavily on sources such as Wikipedia, LinkedIn and Freebase. But they seem to be expanding their data sources on people to other sites and specific pages.

Searching for Simon Le Bon you’ll find that a people snippet appears for Wikipedia, IMDb and Biography. Wikipedia is a no-brainer and IMDb makes a good deal of sense too. Biography was the surprising one.

I noted that both IMDb and Biography had namespaces or folders (underlined in red) that seemed to be easy identifiers for entities. So I decided to look for more sources and I found them. Lots of them.

CrunchBase

MySpace

NBA.com

Quora

TED

ESPN

The Canadian Encyclopedia

Amazon

MTV

Last.fm

Forbes

NNDB

Facebook

Twitter

Yahoo! Movies

Hollywood.com

AskMen

FriendFeed

TV Guide

Comedy Central

Most of these either have a namespace that makes it easy to identify as a person or are clear profiles in the case of MySpace and FriendFeed. Whether it’s ‘player’, ‘artist’, ‘celebrities’, ‘person’, ‘profiles’ or ‘speakers’ it seems like Bing has determined pages that match these specific entities.

About Pages

People snippets show up far more often on about pages which supports the idea that Bing is looking for high confidence entity pages and not assigning real authorship.

As you can see I get a people snippet on my about page but not on my site as a whole. Nor do I get it returned on any of my content. Here’s another example.

Again, the about page on Matthew Inman’s now defunct site is given a people snippet while the site as a whole isn’t. The people snippet is showing pages about that entity, not authored by that entity. It just so happens that there’s some overlap in those areas.

Sorta Structured Data

Many of these pages have a rich amount of data on them. While they aren’t marked-up with any structured data per se, search engines can clearly parse and use that information. Here’s a people snippet via Green Day Authority.

That page has no structured data mark-up but it has structure.

Characters?

Further pushing on the choice of pages to use to apply the people snippet I began to search for characters. First Harry Potter and then Derek Zoolander.

No people snippets are applied even though it’s still pulling from IMDb. The difference here is that it’s plucking out a title page and a character page instead. Maybe that’s not how it works but that’s how my pattern matching mind sees it right now.

[Update 3/21/13]

ChaosSEO noted that he could get character names to render people snippets. Sure enough, you can.

And …

I tend to think that there’s some special casing going on with IMDb so that it only applies the snippet to the name pages, but if you get a snippet to render for an IMBb character page please let me know.

Going through characters was actually really instructive. First I began to see that there were associations between the entities of person and character.

A search for Hermione Granger produces people snippets and a result for Emma Watson. Clearly there’s some understanding that the two are related. You can get that same dynamic for a number of character searches such as Gandalf or Chewbacca.

Finally, I found a result that makes me very confident that this is not authorship at all but entity detection.

Clearly Harrison Ford (or Han Solo) is not the author of these pieces but the subject of them.

Decentralized Images

There are few instances where a site will get a people snippet. This seems to be rare and only occurs when Bing has high confidence that they have the right person.

Here we can see that a people snippet is applied to Seth’s site but that the image is pulled from that site and not from some central database. This provides some variety in what is displayed but also leads to some errors from time to time.

So Bing seems confident that they have the right person associated with that site but the image they pulled is not Jason. It’s Wesley Chan.

Authorship?

Is this a form of Authorship? Sorta, kinda, not really. Sometimes you’ll see what looks like a people snippet pop up on a content page.

Tim is the author of that piece so there’s a chance that they’ve identified and are trying to present Authorship based on that fact. But it’s more likely they just identified him as an entity. Because the images are decentralized it pulls what it can from that article.

Same thing happens with this piece by Mathew Ingram. In both cases there is structured data on that page that would indicate that each is the author of that piece (though they both don’t have Google Authorship working.)

So it’s not true authorship and very few pieces of content have a people snippet right now but if Bing decides to follow this path, make the connections with all of their datasets (including their social sidebar results) then you could see Bing being a legitimate authorship platform.

Right now it seems like the snippets on content results are more like a side effect of people identification.

TL;DR

Bing has introduced people snippets that look like Google’s Authorship snippets but are more focused on identifying people as entities through a variety of sources rather than assigning authorship of content. For now.

It’s been a frustrating few weeks of discussion about Authorship and Author Rank.

Here I will present a few things that may give you some more context overall and, in particular, my point of view on things.

Social Computing Research

Just the other day Google revealed that it gave $1.2 million dollars in awards to those undertaking social computing research.

We know that interactions on the Web are diverse and people-centered. Google now enables social interactions to occur across many of our products, from Google+ to Search to YouTube. To understand the future of this socially connected web, we need to investigate fundamental patterns, design principles, and laws that shape and govern these social interactions.

We envision research at the intersection of disciplines including Computer Science, Human-Computer Interaction (HCI), Social Science, Social Psychology, Machine Learning, Big Data Analytics, Statistics and Economics. These fields are central to the study of how social interactions work, particularly driven by new sources of data, for example, open data sets from Web2.0 and social media sites, government databases, crowdsourcing, new survey techniques, and crisis management data collections. New techniques from network science and computational modeling, social network and sentiment analysis, application of statistical and machine learning, as well as theories from evolutionary theory, physics, and information theory, are actively being used in social interaction research.

We’re pleased to announce that Google has awarded over $1.2 million dollars to support the Social Interactions Research Awards, which are given to university research groups doing work in social computing and interactions. Research topics range from crowdsourcing, social annotations, a social media behavioral study, social learning, conversation curation, and scientific studies of how to start online communities.

What this says to me is that Google is intensely interested in understanding how to use social interaction data. But they’re not there yet. And why should they be? They’ve been working on link based signals and refinement for over 10 years but haven’t delved into social data until the last few.

This is a discipline that they are far from fully understanding. I can’t help but pick out words like ‘investigate’, ‘envision’ and ‘new’. This is a post about the exploration of the effects of social interaction on a host of fields. These are not papers as to their conclusions.

But we do have a few of those papers, areas where Google has begun to learn about how social interactions or signals might impact search. Lets take social annotations as an example.

Social Annotations in Web Search

This research was presented at the 2012 Conference on Human Factors in Computing Systems.

Remember when our SERPs had a whole bunch of smaller faces in them and other various social gestures? Well, Google found that those didn’t work. We hardly noticed them and when we did we didn’t always believe they added value.

In fact, the only thing that really did was the Authorship snippet. It’s a very interesting read if you’re interested in design and authority. The way we see results today is clearly influenced by this research and you can see Google learning more about how social connections and expertise work within search.

This study revealed a counter-intuitive result. Despite having the names and faces of familiar people, and despite being intended to be noticeable to searchers, subjects for the most part did not pay attention to the social annotations.

Our questions about contact closeness, expertise, and topic were answered by the reactions captured during the retrospective interviews. These interviews revealed the importance of contact expertise and closeness, and the importance of the search topics in determining whether social signals are useful, thus echoing past findings on the role of expertise in social search.

I walk away thinking that all of this is much tougher than we believe peering in from the outside. That and Google is at the start of this research, not the end.

Knowing that they aspire to understand these dynamics also makes the closure of Google Reader odd since there is a substantial amount of data that could be mined there, all tied back to identity and, by extension, topical expertise.

Whisper Down The Lane

I had a chance to speak on a panel at SMX West with Mike Arnesen and Lisa Weinberger about Authorship, Author Rank and Authority.

Overall, authorship and the potential for Author Rank was a hot topic that spilled out into multiple other sessions. Both Matt Cutts and Duane Forrester were asked about link based signals versus social signals. You could tell they are both tired of this question. Paraphrasing, they essentially said that while social signals are intriguing they’re not nearly as far along as we in the industry might believe (or want).

When prodded about the collapse of the link graph they noted that the link graph was just fine thank you very much. Link manipulation, the intent behind linking that we feel is so perverted, is not nearly as rampant as we assume. The mainstream blogger or site owner is linking for the right reasons. In short, the link graph is still valuable and with lower friction to producing digital content it may actually improve as more laypeople become content producers.

That’s not to say that social signals aren’t important but it will be a complement to or a refinement of the link graph, not a replacement. This is something I discussed in my original Author Rank post.

If we believe that search engines still view the link graph as viable there may be ways to simply use Authorship to make the link graph more accurate. Think of Authorship as meta information passed on every link. When looking for information on cancer the link given to an article from an established oncologist at a world renowned hospital would likely confer more value than a link to an article from ‘screwcancer888’ at a Q&A site.

In some ways this reminds me of delegating authority which Bill Slawski (always insightful) wrote about back in late 2010. What we’re really talking about is identifying expertise and allowing those experts to help curate our view of those topics where it matters – in search results.

Parsing Statements

It’s enticing to pick apart responses and statement by Googlers when they are asked to comment on Author Rank. The fact is that they’re not going to divulge much or commit one way or the other (at least publicly). They’ve been burned before by saying something that is true but interpreted in different ways.

So when asked, of course they’re going to reply that it’s something they’re experimenting with (because they do aspire to use the data) but that it is currently not a direct ranking signal and nothing to worry about now.

Of course that leads everyone to look for the experiments, to look for indirect ranking signals and to take the ‘now’ as a declaration of sorts for future implementation.

Authorship could be an indirect signal if you believe (like I do) that the click through rate (CTR) on a result can provide a positive feedback signal. And we know the CTR on authored results disrupts the normal click distribution of a SERP. Of course Google could take into account the Authorship snippet and normalize the CTR impact. So perhaps it isn’t having that indirect impact. See how confusing it can get?

Just for fun, let us think what would transpire if a Googler simply said there is no such thing as Author Rank without any hedging or caveats. People would start to conflate that with Authorship, potentially reducing the adoption rate. Many would interpret it to mean that Google had abandoned author based weighting completely. Thus, when Google did figure it out and apply it the industry would point to the statement and shout ‘liar’ at the top of their lungs.

We’ve trained Google to provide us with these elliptical statements. I choose to view them through this lens.

What To Look For?

That’s not to say that we shouldn’t be interested in the topic. I like the testing Terry Simmonds is doing on the mechanics of Authorship because it documents how Google is trying to extend the mark-up to more of the content on the web. And that’s a constraint as far as I can tell right now. Conversations about the inability to roll out updates because of low adoption are not uncommon.

You can’t begin to rank results based on topical expertise if many of the experts aren’t included in the selection criteria. The participation rate in Authorship has to be such that using it would provide a materially better ranking of content. Reports have Authorship coverage as low as 9% and as high as 17%. That’s not a lot really and both studies are limited based on the relatively small data sets analyzed.

The problem? If you were to want information on astrophysics you’d probably want to include Neil deGrasse Tyson in those results. Yet, he’s not on Google+ (as far as I can tell) and isn’t part of the Authorship program.

Looking at how Google is trying to assign Authorship is important.

The mechanics and the indirect Authorship Google often grants is particularly intriguing. I noted that Jonathon Colman was receiving a bounce back Authorship link on a SlideShare URL for which no direct Authorship mark-up was present.

I recall seeing this in the past on URLs from Quora, FriendFeed and Flickr. I swear some of these used to show up in Author Stats but I haven’t seen them lately (except for FriendFeed which I see at the tail end of my list.)

In fact, the bug that took Author Stats down might have been the exposure of indirect Authorship based on high confidence in matching public social graph data to Google+ profiles. Rapleaf got the brunt of the ire for crawling the public social graph but Google clearly has and continues to use this information even though the social circles feature has been retired.

Looking today I see another interesting URL showing up in Author Stats – Twitter.

There’s quite a lot of evidence that Twitter is a fairly well trusted source of indirect Authorship, but that’s a post for another day. However, we can also look at the verbiage in the Structured Data Testing Tool, which has changed within the last few weeks.

The points of interest here are the ‘(direct or indirect)’ verbiage as well as the fact that the tool only checks the first rel=author link listed on a webpage.  The former certainly makes me believe that assigning Authorship based on indirect links is important to Google.

The latter tells me two things. First that the tool should not be trusted as the final arbiter of whether the correct Authorship is or will be applied. Second that Google obviously sees multiple authors or entities (or agents) on the page.

Lets go a step further. Google’s new Social Sign-In can be construed as a portable digital signature which might allow Google to rely on comments and other content produced outside of Google+. So tracking how this is rolled out and whether the reviews that now flow under your profile are also granted Authorship are interesting developments.

I’ve been eager to see Author Rank implemented since I first saw Matt Cutts interview Steven Levy.

This actually predates Authorship and the follow-up question by Matt (along with a bit of body language) makes it clear that Google was thinking about this seriously. While I absolutely do look for connections and patterns that might paint a picture of the future I’m not looking for it behind every corner and trying to fit Author Rank into each and every odd result or anecdote.

Authority

I prefer to talk about how people might build authority rather than how they would build Author Rank. Just as links are the result and not the goal, Author Rank will be the result and not the goal of your efforts.

Discussions about what makes someone an authority and how Google might want to translate that into math are fascinating. What makes someone authoritative versus popular? Is there a difference? If so, how would you go about separating the two?

How do you map the decline of authority? Of someone who is no longer really an expert and just mailing it in? Can you identify this even if they remain popular? How can you tell if someone is endorsing content based on merit or friendship? Is it what you know or who you know?

Furthermore, you could find that one was popular for the wrong reasons. Would you want to rank someone highly who simply fanned the flames of dissent and created controversy? The tone and type of interaction will be important so sentiment analysis and other processes will need to determine how to use social interaction as a reliable signal.

Influence

And how does influence fit into this equation? One can be influential without being popular, but clearly being popular gives you a better chance of being influential just by sheer reach. Can you be influential without being an authority? I think so. Just look at Jenny McCarthy and her influence within the anti-vaccine movement.

The latter clearly strays into the subjective nature of quality, relevance and authority that I touched on after the Panda update. Personalization helps to ensure that your subjective view of authority is reflected back to you. That’s why search results are changed based on who you follow on Google+. And personalization of search results is the most important thing about Google+ in my view.

But in discussing how Google might identify authority and expertise, we’re dealing with the aggregate. So the question isn’t really about your personal view (which is reflected back in Search+ results) but how the aggregate views different figures and authorities.

Of course, being likable is part of the way you can obtain authority. And it is often not what you say, but how you say it (or present it) that gets you noticed. So part of building authority is in ensuring that you can communicate in a way that conveys that expertise but also makes it accessible and … memorable.

Yes, I see all of this as being related because the same content presented in comic sans without any images or paragraph breaks wouldn’t have nearly the same impact and would not, ultimately, convey authority. Even though the actual words are the same!

I had a similar conversation with Dan Shure where he wondered about the impact of publishing content from Rand Fiskin under somebody else’s name. Would it get as much ‘play’ and be received as well? I doubt it. So what does that say about the connection of authority, popularity and quality assessment?

These are just a few of the things that make this topic so incredible.

TL;DR

I believe Google wants to use Author Rank but I also believe that it’s far more difficult than we think. Focusing solely on Author Rank may blind us to tracking Google’s progress and building what is truly important. Authority.

I’m a dedicated Google Reader user, spending hours each day using it to keep up on any number of topics. So my knee-jerk reaction to the news that Google will close the service as of July 1, 2013 was one of shock and anger.

I immediately Tweeted #savegooglereader and posted on Google+ in hopes of getting it to trend or go hot. These things are silly in the scheme of things. But what else is there to do?

I’ve written previously that the problem with RSS readers is marketing. I still believe that (it’s TiVo for web content people!) but in the end that’s not why closing Google Reader is so dangerous. And it is dangerous.

Google Reader Fuels Social

Photo via double-h

The announcement indicates that, while having a loyal following, usage has declined. That’s a rather nebulous statement, though I don’t truly expect Google to provide the exact statistics. But it’s who is still using Google Reader that is important, is it not?

Participation inequality, often called the 90-9-1 principle, should be an important factor in analyzing Google Reader usage. Even if you believe that the inequality isn’t as pronounced today, those that are contributing are still a small bunch.

Studies on participation on Twitter have shown this to be true, both from what content is shared and who is sharing it. That means that the majority of the content shared is still from major publications and that we get that information through influencers. But where do they get it?

Google Reader.

RSS readers are the snowpack of social networks.

Organizing Information

Google’s mission is to organize the world’s information and make it universally accessible and useful. By extension that is what Google Reader lets power-users do. Make no mistake, Google Reader is not a mainstream product. Google (and many others) have screwed up how to market time-shifted online reading.

The result is that those using Google Reader are different. They’re the information consumers. They’re the ones sifting through the content (organizing) and sharing it with their community (accessible) on platforms like Twitter, Facebook and Google+ (useful).

Google Reader allows a specific set of people to help Google fulfill their mission.

Losing Identity

There are replacements to Google Reader such as Feedly. So you can expect that the people who fuel social networks will find other ways to obtain and digest information so they can filter it for their followers. Problem solved, right? Wrong.

Why exactly does Google want to hand over this important part of the ecosystem to someone else? With Google Reader they know who I am, what feeds I subscribe to, which ones I read and then which ones I wind up sharing on Google+.

Wouldn’t knowing that dynamic, of understanding how people evaluate content and determine what is worthy of sharing, be of interest to Google? It should be. It’s sort of what they want to excel at.

Not only that but because Google Reader has product market fit (see how I got that buzzword in there) with influencers or experts, you’re losing an important piece of the puzzle if you’re thinking about using social sharing and Authorship as search signals.

Data Blind

In the end, I’m surprised because it makes Google data blind. As I look at Unicorn, Facebook’s new inverted-index system, I can’t help but think that Facebook would love to have this information. Mining the connections and activity between these nodes seems messy but important.

What feeds do I subscribe to? That social gesture could be called a Like in some ways. What feeds do I read? That’s a different level of engagement and could even be measured by dwell time. What feeds and specific content do I share? These are the things that I am endorsing and promoting.

By having Google Reader integrated into the Google+ ecosystem, they can tell when I consumed that information and when I then shared it, not just on Google+ but on other platforms if Google is following the public social graph (which we all know they are.)

Without Google Reader, Google loses all of that data and only sees what is ultimately shared publicly. Never mind the idea that Google Reader might be powering dark social which could connect and inform influencers. Gone is that bit of insight too.

Multi-Channel Social

As a marketer I’m consumed with attribution and Google Analytics clearly understands the importance of multi-channel modeling. We even see the view-through metric in Google Adwords display campaigns.

The original source and exposure of content is of huge importance. Google might have Ripples but that only tells them how the content finally entered Google+ not how that content was discovered.

I’m certain that users will find alternatives because there is a need for this service. Google just won’t know what new sites influencers might be reading more of or which sites might be waning with subject matter experts. Google will only see the trailing indicators, not the leading ones.

TL;DR

Google Reader allows information consumers – influencers and subject matter experts – to fuel social networks and help fulfill Google’s core mission. Closing Google Reader will put that assistance in the hands of another company or companies and blinds Google to human evaluation data for an important set of users.

2012 was a fantastic year for Blind Five Year Old. I met most of my goals, came to a few epiphanies but often found it difficult to juggle everything at once. In all, this is what I learned.

Stop Comparing

There are a number of ‘names’ in the SEO community and there’s a growing trend to share your journey – to open the kimono so to speak. (Sort of like what I’m doing here which is going to be strange given my next statement.) The odd thing about this transparency is that it puts a bit of pressure on others. Or maybe that’s just me.

I had a chance to sit and chat with Wil Reynolds. I talked with Rhea Drysdale. They were generous with their time and gave a lot of excellent advice. Yet for a brief while those conversations also made me feel pretty lousy.

I started wondering. Was I doing enough to build my company? Was I falling behind? After establishing myself and building my brand was I frittering it away? Would I just be a ‘lifestyle business’? Shouldn’t I get bigger and build an agency? Are they so much better at this stuff than me?

I came to realize that I wasn’t enjoying my success. And that sucked, particularly because I was doing really well. So I decided to stop comparing my journey to those of others.

I am not Wil or Rhea or Rand. They all provide amazing advice based on their journey and personal situations. Mine will be different because I’m different. Hopefully I’ll learn from their insight and experiences but I’m sure I’ll make some of the same mistakes they made as well as others as I find my way. And that’s okay.

Take Risks

One of my goals was to speak at two conferences in 2012. Mind you, I’d never spoken at a conference before and while I’ve done plenty of trainings in front of large groups this would definitely be out of my comfort zone. I’m still an introvert at heart.

I snagged a spot at SMX West 2012 to speak about Authorship. I worked on that deck for ages. I obsessed over it. Then I found out that the presenter notes wouldn’t be available. Yikes! I was incredibly nervous but there were people like Aaron Friedman and Nick Roshon who were eager to see me present and gave me encouragement.

I was up there on stage with Dennis Goedegebuure. He’s done a lot of speaking and seemed … unflappable. “Do you still get nervous?” I asked him. “Oh yeah, every time” he replied. That made me feel better and helped me more than he probably realized. The presentation was … okay. I think I read a bit too much, had slide problems and went long, which meant Vanessa was about ready to shove me off the stage.

It was done. It had gone well enough. People didn’t heckle me and there weren’t any Tweets about how much I sucked. The world kept spinning. I needed that experience because the next time I presented was at MozCon 2012 in front of about 800 or so people. Crazy! I’m not sure Rand knew this would only be my second presentation or not but I’m very thankful for the opportunity he provided.

With the help of some amazing advice I was able to build a much better deck this time. I was a total and complete wreck before I presented. So if you met me there before my slot I might have seemed a bit preoccupied. (A thank you to Mackenzie Fogelson, Pete Meyers and Cyrus Shepard for distracting me with interesting conversations.)

I think I did well. It felt … good, which was an odd sensation for me. And the feedback and score I received validated my effort.

I’ve always taken risks throughout my career and that has to continue if I’m going to grow.

Retain Confidence

I’ve had a crisis of confidence a few times in the past, mostly brought on by my own harsh criticism. That didn’t happen this year but between comparing myself to others and working myself up into a lather about presenting, I may have had a few doubts here and there.

But you have to kick those gremlins out of your head. Confidence is so important. Don’t confuse that with being a cocky douchebag. Confidence simply means that you know you’ve done everything you can do and that you’re comfortable with what you’re putting out there. It’s also acknowledging that you’re not always going to be right. That’s life so get used to it and move on.

This piece from Todd Mintz was brave and worth reading. Todd’s a smart and talented guy but he gets smarter and more skilled as time goes by. The post shows that we can only be confident about where we are at any given point in time. Will we make errors? Sure. But we learn from them and get better. Don’t look back and let mistakes sap your confidence, let it fuel it instead.

Keep Learning

 

In this industry you simply must keep learning. My definition of SEO is quite broad, which means that I need to know a little bit about everything.

Everything is a lot! Some of it you’re not going to understand at first but you have to keep pushing. Ask questions, even dumb ones. Just keep picking up new skills and experimenting. I can not stress enough how beneficial experiential learning is in this business. Don’t just take my word or some expert’s word on how something works, try it yourself.

Because we’re in a post modern SEO era.

Postmodern SEO develop strategies and tactics based on individual context, not on preconceived “Best Practices,” or some blogger’s interpretation of “standards.” Instead we consider things like business goals, risk, longevity, audience and others to build individual strategies.

Do. The. Work.

Watch The Clock

There are simply not enough hours in the day. Success has been great but it also means I’m juggling a lot more. I’ve got more clients. I’ve got a part-time writing gig at Marketing Land. I’m speaking at conferences. I’m keeping up on industry news. And the email just never stops.

I don’t expect anyone to feel sorry for me. That would be ludicrous! These are good problems. But I haven’t quite mastered how to balance everything yet. I’ve contemplated stopping my #ididnotwakeupin series. I’ve missed out on requests to contribute to articles. Sometimes things just fall through the cracks. And I hate that.

Through it all I have guarded my personal time. I’m still working more than I ever have, but I don’t pull that many crazy hours. I take the time to build Legos with my daughter, play family bingo games or games of Ticket To Ride, watch episodes of Nikita or just have an afternoon off with my wife.

Love Your Calendar

The primary way I began to take back control of my time was to rely on my calendar. I started to put everything in my Google Calendar, including all those ‘tentative’ meetings. Because the worst thing that can happen is you tell three people you’re available on a certain day and within the span of a few hours they all try to book the same time.

Not only were there fewer missed connections but I was able to see the time I had available for other work. It became more and more clear that I had to book hours to do the work too.

Keep Fit

I also made time to workout. I lost 30 pounds and kept it off by counting calories and working our regularly. I admit, part of this was driven by pure vanity. I didn’t want to stand up in front of a lot of people and look bad.

Besides the obvious health benefits, the other reason was also selfish. Staying fit made me a better thinker. Working out let me clear my head and afterwards I was definitely sharper. I think of working out a little like being organized. It takes a bit of time each week but it makes me a lot more productive.

Ditch False Modesty

I ran into Marty Weintraub at both SMX West and MozCon. It was at the latter that he basically called me out. He complimented me on my presentation and I did the ‘aw shucks, thank you, just trying my best’ routine and he told me to stop with the false modesty and instead simply say thank you and accept the praise.

That doesn’t come naturally to me but it was a turning point. I needed to embrace those who appreciated me. I mean, there are going to be plenty of folks who try to tear you down in life so when you’re recognized as being good at something just run with it.

Overcome Guilt

That image will give you a headache right? And that’s the point I’m making here. Guilt is awful but I’ve got a lot of it.

I don’t have much guilt about ‘making it’. I worked hard and put in a lot of time and effort. But I recognize that I didn’t do it alone. I was helped by many many people along the way. So I try to do the same. But that’s not always easy. I despair when I don’t get back to someone’s email or Google+ post.

I even have some guilt about mentioning some people in this post but not others. How can I leave out people like Matt McGee, Anthony Pensabene, Jon Henshaw, Bill Sebald, Zeph Snapp, Max Minzer and Tadeusz Szewczyk.

And I’m leaving a ton of other people out here! I don’t want to slight anyone. I want to acknowledge their contribution and worth. I value my Google+ community. I care. A lot. Yet it’s nearly impossible for me to communicate that. So I’m letting go of that guilt little by little.

Yet, I doubt I’ll get rid of all my guilt because I think it makes me a better person.

Battle Perfectionism

Am I a perfectionist? If you have to ask yourself that question I think you’re likely closer to one than you might think. I have very high standards and I like to present things when I have pulled on every little thread and packaged it up into something that is appealing as well as informative.

This wreaks havoc with my time management and I try to live by the ‘done is better than perfect’ mantra. I nod my head when Jonathon Colman talks about it and often give this exact advice to others. Yet, I find it tough to follow in practice.

The reason why is that my quest for superior quality at all costs has netted me a really nice referral business. I know I should give myself a break but I fear the slippery slope of sloppy work.

Yet I’m beginning to see a light at the end of the tunnel as I work on some other projects and collaborate in different ways. That said, don’t expect this to become a high volume blog … ever.

Embrace The Unknown

I remember when I would interview for a job and I’d get that ‘where do you see yourself in 5 years’ question. Based on my life experience I was usually honest in telling people I had no idea. Shit happened and you just never could know how things would turn out. You can only open the door right in front of you and see where it goes.

So I don’t know how Blind Five Year Old will grow, though I think it will. I don’t know what new things I’ll be doing this year. Maybe I’ll build a product. Maybe I’ll do more writing. Maybe I’ll write a book. I just don’t know yet and I’m okay with that.

It’s not that I’m not ambitious or that I don’t have goals. I am and I do. It’s just figuring out which direction to go and opening that door.

Google’s evil plan is simple and not so evil.

Don’t Be Evil

Any successful company is going to draw criticism. Google probably gets more of it than others because of their ‘Don’t Be Evil’ motto. Algorithm changes shuffle branded sites higher and people shout ‘evil!’ Google begins to disintermediate certain verticals and people shout ‘evil!’

Most of the posts about Google’s evil ways focus around these two themes. So much time and energy is spent raging against changes that are simply a reflection of us – the user. When we collectively stop shopping at branded stores over smaller boutiques then we’ll see that reflected in our search results.

And the last time I checked no one was mourning the demise of the milk man or shedding tears over Tower Records or Blockbuster. It sucks if you’re the business getting disintermediated but do you really want to go to another website to get the current weather?

Evil? It’s not Google, it’s you.

Google’s Evil Plan

Instead of talking about all of these natural business moves and conjuring up some nefarious plot, I want to talk about Google’s real strategy. Here’s the truth. Here’s Google’s plan.

Get people to use the Internet more.

That’s it. The more time people spend on the Internet the more time they’ll engage in revenue generating activities such as viewing and clicking display ads and performing searches.

The way Google executes on this strategy is to improve speed and accessibility to the Internet. Google wants to shorten the distance between any activity and the Internet. Lets look at some of Google’s initiatives with this in mind.

Chrome

Firefox was doing a bang up job of breaking Internet Explorer’s browser monopoly. Chrome certainly hastened IE’s decline and helped secure more search volume. Yet Chrome developers have long said that their goal isn’t market share but to make the browsing experience faster.

In a very nearsighted way, making browsers faster is the goal. Yet, the faster the web experience, the more page views people rack up and the more searches they’ll perform.

Chrome is about reducing the friction of browsing the Internet.

SPDY

Google can only do so much with Chrome to speed up the web. Enter SPDY, an open networking protocol, which looks to be the basis for HTTP 2.0.

Its goal is to reduce the latency of web pages.

That’s technical speak for making the web faster. This is what users want. This is what makes users happy. Milliseconds matter when it comes to user satisfaction. And satisfying the user is great for business.

Android

Similar to Chrome, Google saw that users would increasingly access the Internet via phones. They learned from their web browser experience and decided to jump into the vertical early and it’s paid off. Google now commands nearly 54% of the smartphone market.

Android doesn’t have to make money directly. It provides unfettered access to revenue generating activities and allows Google to push the industry forward in terms of speed.

Motorola Mobility

Not content to simply push the envelope with software, Google decided to grab Motorola Mobility and improve on hardware too. The rumors around the Google X phone are increasing.

Long battery life and wireless charging are two of the more tantalizing possibilities  These are clearly features that would greatly benefit users but … they also ensure that you’ll nearly always be able to connect to the Internet. See how that works?

Google Now

Not using the Internet enough? Google Now can help change that by automagically serving up useful cards based on your search history and behavior. Don’t get me wrong. I like Google Now and find it to be more and more valuable as they add more functionality.

But it’s no mystery that predictive search is also about stimulating more Internet activity.

Google Fiber

Many seem to think Google is crazy to pursue fiber. It’s massive. It’s expensive. But it’s also exactly in line with their goal of increasing Internet usage. In fact, they’re pretty clear in the messaging on the Google Fiber page.

Google Fiber starts with a connection speed 100 times faster than today’s broadband. Instant downloads. Crystal clear high definition TV. And endless possibilities. It’s not cable. And it’s not just Internet. It’s Google Fiber.

It’s not that Google would control the transmission (though that’s a nice side benefit), it’s that the friction to using the Internet would be nearly zero.

Google WiFi

 

Google already provides free WiFi in Mountain View, wanted to do it in San Francisco as far back as 2005 until it was torpedoed by politics and paranoia. Now Google provides free WiFi in the Chelsea neighborhood of New York. In addition, Google has been futzing with white space and a super-dense LTE network.

Can it be any more clear? Google wants ubiquitous Internet access.

Google Drive

I often see people argue that the cloud is Google’s big picture strategy. I think that’s still missing the point. The cloud is a means to an end.

Giving people the ability to access files from anywhere simply keeps them online longer. You don’t have the browser off working on your document, instead your online editing and saving your document. You’re searching for those documents.

You’re just a browser tab away from areas of the Internet where Google makes money. In short, Google Drive shortens the distance between work and activities that produce revenue for Google.

Chromebook

Taken to the extreme, Chromebook is essentially a computer that runs off the Internet and cloud. Everything is done online.

A new type of computer designed to help you get things done faster and easier.

Faster. There’s that word again. And easier is just a friendly way of saying ‘reduce friction’. At $199 and $249 Google is hoping that this new type of computer will start to find a market. This strikes me as the ultimate lock-in.

Google+

So what about Google+? At first blush, it doesn’t seem to fit.

I still believe a substantial reason for building Google+ was to develop better social signals and increase search personalization. However, I think the time spent in places where Google couldn’t reach (aka Facebook) was troubling.

Google needed to break the stranglehold Facebook had on social attention. They’ve certainly made inroads there and that’s really all they needed to do to ensure attention didn’t pool and persist in a Google dead zone.

Self Driving Cars

I’m shocked that people don’t see the brilliance of a self-driving car. The average commute time in the US is 25 minutes (pdf). So that’s nearly an hour each day that people can’t be actively on the Internet. Yet, they obviously want to be.

If you play Ingress (like I do) you can see where XM (roughly phone usage) is highest. It’s super high in parks and doctor’s offices and movie theaters. But it’s also concentrated at intersections. A red light and we’re diving for our phones.

Now imagine a self-driving car and how much more time you’d have to … be on the Internet. I’m just talking about commuting which is less than 20% of the driving done in this country!

A self driving car unlocks a vast amount of time that could be spent on the Internet.

Google Glass

I know the latest big thing is Sergey on the Subway but to me his skydive was more transformative. The message? Even if you’re falling out of the sky you can still use the Internet.

Google Glass could be the ultimate way to keep you connected to the Internet.

Perhaps we’ll reach a point where much of our consciousness is actually online. Why waste your time remembering useless things when you can simply retrieve them from your personal cloud? Sometimes the future in Charles Stross’ Accelerando seems almost inevitable.

Mind you, at times I feel the urge to live in a cabin in the woods but it’s usually quickly followed with a caveat of ‘with good satellite coverage or Internet access.’

Google TV

I think YouTube was initially thought to be the future of TV. The problem is that we’re very entrenched in traditional TV and inertia (and a lack of proper execution by Google TV) has allowed traditional TV to catch up.

This is the one place where Google is behind. Maybe Google TV picks up steam, or Google Fiber is the wedge into homes or Google acquires someone big like TiVo or Netflix.

Twitter is also both a major rival and potential acquisition target because of their position as the glue between screens.

Share of Time

I’m surprised that no one has compared Google’s strategy to Coke’s now abandoned ‘share of stomach‘ strategy. Google wants people to spend more of their time on the Internet. Think about that.

Once again it comes down to the ‘Don’t Be Evil’ motto. Coke didn’t care if they were creating a health epidemic as they rang up profits. Google, on the other hand, believes their services can improve our lives.

That kind of belief is what the tin foil hat conspiracy folks should really be worried about. It’s not any small tactical gaffe that could be chalked up in the evil column. It’s that Google believes they’re doing good. I sort of think so too.

TL;DR

Google’s strategy is to get people to use the Internet more. The more time people spend on the Internet the more time they’ll engage in revenue generating activities. As such, nearly every Google effort is focused on increasing Internet speed and access with the goal to shorten the distance between any activity and the Internet.

Tracking keyword rank is as old as the SEO industry itself. But how you do (and use) it is changing. Are you keeping up?

This post covers how I create and use rank indexes and introduces a new and improved way to track rank in Google Analytics.

Rankaggedon

In December of 2012 both Raven and Ahrefs made the decision to shut down their rank tracking features because they violated Google’s Terms of Service. The reaction from the SEO industry was predictable.

The debate about why Google began to enforce the TOS (I think it has to do with the FTC investigation) and the moaning about how unfair it is doesn’t interest me. Both SEOmoz and Authority Labs still offer this service and the way many use rank needs to change anyway.

Every obstacle is an opportunity. Trite but true.

Is Rank Important?

To be honest, I don’t use rank that much in my work. This has to do with a combination of the clients I choose to work with and my philosophy that increasing productive traffic is the true goal.

Yet, you’d have to be soft in the head not to understand that securing a higher rank does produce more traffic. Being on the first page matters. Getting in the top three results can produce significant traffic. Securing the first position is often a huge boon to a business. Duh!

But rank is the extrinsic measurement of your activities. It’s a Google grade. Rank isn’t the goal but the result.

Unfortunately, too many get obsessed with rank for a specific keyword and spend way too much time trying to move it just one position up by any means necessary. They want to figure out what the teacher is going to ask instead of just knowing the material cold.

Rank Indexes

So how do I use rank? I create rank indexes.

A rank index is the aggregate rank of a basket of keywords that represent a type of query class that have an impact on your bottom line. For an eCommerce client you might have a rank index for products and for categories. I often create a rank index for each modifier class I identify for a client.

Usually a rank index will contain between 100 and 200 keywords that represent that query class. The goal is to ensure that those keywords reflect the general movement of that class and that changes in rank overall will translate into productive traffic. There’s no sense in measuring something that doesn’t move your business.

If that rank index moves down (lower is better) then you know your efforts are making a difference.

Executives Love Indexes

A rank index is also a great way to report to C Level executives. These folks understand index funds from an investment perspective. They get this approach and you can steer them away from peppering you with ‘I did this search today and we’re number 4 and I want to be number 1’ emails.

It becomes not about any one term but the aggregate rank of that index. That’s a better conversation to have in my opinion. A rank index keeps the conversation on how to move the business forward instead of moving a specific keyword up. 

Getting Rank Index Data

If you’re using SEOmoz you export the entire keyword ranking history to CSV.

After a bit of easy clean up you should have something that looks like this in Excel.

At this point I simply copy and paste this data into my prior framework. I’ve already configured the data ranges in that framework to be inclusive (i.e. – 50,000 rows) so I know that I can just refresh my pivot table and everything else will automagically update.

If you’re using Authority Labs you’ll want to export a specific date and simply perform the export each week.

There’s a bit more clean up for Authority Labs data but in no time you get a clean four column list.

Unlike the SEOmoz data where you replace the entire data in your framework, you simply append this to the bottom of your data. Once again, you know the pivot table will update because the data range has been configured to be quite large.

Creating The Rank Index Pivot Table

You can review my blow by blow of how to create a pivot table (though I’m not using a new version of Excel so it all looks different anyway.) It’s actually a lot easier now than it was previously which is something of a miracle for Microsoft in my view.

You’ll use the keyword as your row label, date as the column label and the Average of rank as the values. It’s important to use a label so you can create different indexes for different query classes. Even if you only have one index, use a label so you can use it as a filter and get rid of the pesky blank column created by the empty cells in your data range.

You may notice that there are a lot of 100s and that is by design.

All those non-ranked terms need to be counted somehow right? I chose to use 100 because it was easy and because Authority Labs reports up to (and sometimes beyond) that number.

Turning Rank Data Into A Rank Index

Now that you have all the rank data it’s time to create the rank index and associated metrics.

Below the pivot table it’s easy to use a simple AVERAGE function as well as various COUNTIF functions to create these data points. Then you can create pretty dashboard reports.

Average Rank is the one I usually focus on but the others are sometimes useful as well and certainly help clients better understand the situation. A small caveat about the Average Rank. Because you’re tracking non-ranking terms and assigning them a high rank (100) the average rank looks a bit goofy and the movement within that graph can sometimes be quite small. Because of this you may wind up using the Average of Ranking Terms as your presentation graph.

I don’t care much about any individual term as long as the index itself is going in the right direction.

Projecting Traffic

I can always look at the details if I want and I’ve also created a separate tab which includes the expected traffic based on the query volume and rank for each term.

This simply requires you to capture the keyword volume (via Google Adwords), use a click distribution table of your choosing and then do a VLOOKUP.

IFERROR(([Google Adwords Keyword Volume])*(VLOOKUP([Weekly Rank],[SERP Click Distribution Table]),2,0)),0)

You’ll need to divide by 4 to get the weekly volume but at that point you can match that up to real traffic in Google Analytics by creating a regex based advanced segment using the keywords in that index.

Of course, you have to adjust for (not provided) and the iOS attribution issue so this is very far from perfect. And that’s what got me really thinking about whether rank and rank indexes could be relied on as a stable indicator.

What is Rank?

The rise in (not provided) and the discrepancies often seen between reported rank volume and the traffic that shows up point to the increase in personalization. SERPs are no longer as uniform as they once were and personalization is only going to increase over time.

So you might have a ‘neutral’ rank of 2 but your ‘real’ rank (including context and personalization) might be more like a 4 or 5.

That’s why Google Analytics rank tracking seems so attractive, because you can get real world ranking data based on user visits. But that method is limited and makes reporting a huge pain in the ass. The data is there but you can’t easily turn it into information … until now.

Improved Google Analytics Rank Tracking

I got to talking to Justin Cutroni (a really nice and smart guy) about the difficulties around tracking rank in Google Analytics. I showed him how I use rank indexes to better manage SEO efforts and over the course of a conversation (and a number of QA iterations) he figured out a way to deliver keyword rank the way I wanted in Google Analytics.

Using Events and the value attached to it, we’ve been able to create real keyword rank tracking in Google Analytics.

The Avg. Value is calculated by dividing the Event Value by Total Events. You could change this calculation once you do the export to be Event Value by Unique Events if you’re concerned about those users who might refresh the landing page and trigger another Event. I haven’t deployed this on a large site yet to know whether this is a real concern or not. Even if it is, you can always change it in the export.

So you can just make Avg. Value a calculated field and then continue to tweak the exported data so that it’s in a pivot table friendly format. That means adding a date column, retaining the Event Action column but renaming it keyword, adding a Tag column, and retaining the Avg. Value column.

You essentially want it to mimic the four column exports from other providers. I suppose you could keep a bunch of this stuff in there and not use it in the pivot table too. I just like it to be clean.

Event Based Rank Tracking Code

Start tracking rank this way on any Google Analytics enabled site by dropping the following code into your header.

To make it easier, the code can be found and copied at jsFiddle. Get it now!

Just like the old method of tracking rank in Google Analytics, this method relies on finding the cd parameter (which is the actual rank of that clicked result) in the referring URL. This time we’re using Event Tracking to record rank and putting it in a field which treats it as a value.

The code has also been written in a way to ensure it does not impact your bounce rate. So there’s no downside to implementation. You will find the data under the Content > Events section of Google Analytics.

Just click on Content, Top Events and then RankTracker and you’ll find keyword ranking data ready for your review.

Google Analytics Rank Indexes

I’ve been working at applying my index approach using this new Event based Google Analytics rank tracking data. The first thing you’ll need to do is create an advanced segment for each index. You do this by creating a regex of the keywords in that index.

Sometimes you might not get a click on a term that is ranked 20th and certainly not those that are ranked 50th. That’s a constraint of this method but you can still populate an entire list of keywords in that index by doing a simple VLOOKUP.

IFERROR(VLOOKUP(A1,'Export Event Data'!$A$1:$E$5000,5,FALSE),100)

The idea is to find the keyword in your export data and report the rank for that keyword. If the keyword isn’t found, return a value of 100 (or any value you choose). From there it’s just about configuring the data so you can create the pivot table and downstream reports.

Caveats

This new way of tracking is different and has some limitations. So lets deal with those head on instead of creating a grumble-fest.

The coverage isn’t as high as I’d like because of (not provided) and the fact that the cd parameter is still only delivered in about half of the referrers from Google. I’m trying to find out why this is the case and hope that Google decides to deliver the cd parameter in all referrers.

Full coverage would certainly increase the adoption of rank tracking in Google Analytics and reduce those seeking third party scraped solutions, something Google really doesn’t like. It’s in their self-interest to increase the cd parameter coverage.

As an aside, you can get some insight into the rank of (not provided) terms and match those to landing pages, which could be pretty useful.

The other limitation is that you only get the rank for those queries that received clicks. So if you’re building a rank index of terms you want to rank for but aren’t and track it over time it becomes slightly less useful. Though as I’ve shown above you can track the average of ranking terms and of the index as a whole at the same time.

One of the better techniques is to find terms that rank at 11 to 13 and push them up to the front page, usually with some simple on-page optimization. (Yes, seriously, it’s way more effective than you read about.) So this type of tracking might miss a few of these since few people get to page 2 of results. Then again, if you see a rank of 11 for a term with this tracking that’s an even higher signal that getting that content to the front page could be valuable.

Finally, the data configuration is, admittedly, a bit more difficult so you’re working a tad harder to get this data. But on the other hand you’re seeing ranking data from real users. This could get really interesting as you apply geographic based advanced segments. Larger organizations with multiple locations might be able to determine which geographies they rank well in versus those where they’re struggling.

And not Or

At this point I can’t say that I’d scrap traditional rank tracking techniques altogether, though I’m sure Google would like me to say as much. Instead, I think you should use the new Google Analytics Event Based Rank Tracking in conjunction with other ranking tools.

First off, it’s free. So there’s no reason not to start using it. Second, you get to see real world rank, which while limited in scope can be used to compare against neutral rank offerings. Lastly, if you’re trying to future proof your efforts you need to be prepared for the potential end to traditional ranking tools or such high variation in personalization to make them unreliable.

Did I mention this new rank tracking method is free?

I’m looking forward to putting this into practice and comparing one tracking method to the other. Then we’ll see the potential variance between personalized ranking versus anonymized ranking.

TL;DR

The closure of recent third-party rank tracking services is an opportunity to think about rank in a different way. Using a rank index can help keep you focused on moving the business forward instead of a specific keyword. To future proof your efforts you should implement improved Google Analytics rank tracking for free.

I’ve made predictions for the past four years (2009, 2010, 2011, 2012) and think I’ve done pretty well as a prognosticator.

I’m sometimes off by a year or two and many of my predictions are wrong where my predictions were more like personal wishes. But it’s interesting to put a stake in the ground so you can look back later.

2013 Predictions

Mobile Payment Adoption Soars

If you follow my Marketing Biz column you know I’m following the mobile payments space closely. Research seems to indicate that adoption of mobile payments will take some time in the US based on current attitudes.

I believe smartphone penetration and the acceptance of other similar payments such as app store purchases and Amazon Video on Demand will smooth the way for accelerated mobile payment adoption. Who wins in this space? I’m still betting on Google Wallet.

Infographics Jump The Shark

Frankly, I think this has already happened but perhaps it’s just me. So I’m going to say I’m the canary in the coal mine and in 2013 everyone else will get sick and tired of the glut of bad Infographics.

Foursquare Goes Big

The quirky gamification location startup that was all about badges and mayorships is growing up into a mature local search portal. I expect to see Foursquare connect more dots in 2013, making Yelp very nervous and pissing off Facebook who will break their partnership when they figure out that Foursquare is eating their local lunch.

Predictive Search Arrives

Google Now is a monster. The ability to access your location and search history, combined with personal preferences allows Google to predict your information needs. Anyone thinking about local optimization should be watching this very closely.

Meme Comments

A new form of comments and micro-blogging will emerge where the entire conversation is meme based. Similar to BuzzFeed’s reactions, users will be able to access a database of meme images, perhaps powered by Know Your Meme, to respond and converse.

Search Personalization Skyrockets

Despite the clamor from filter bubble and privacy hawks, Google will continue to increase search personalization in 2013. They’ll do this through context, search history, connected accounts (Gmail field trial) and Google+.

The end result will be an ever decreasing uniformity in search results and potential false positives in many rank tracking products.

Curation Marketing

Not content with the seemingly endless debate of SEO versus Inbound Marketing versus Content Marketing versus Growth Hacking we’ll soon have another buzzword entering the fray.

Curation marketing will become increasingly popular as a way to establish expertise and authority. Like all things, only a few will do it the right way and the rest will be akin to scraped content.

Twitter Rakes It In 

I’ve been hard on Twitter in the past and for good reason. But in 2013 Twitter will finally become a massive money maker as it becomes the connection in our new multi-screen world. As I wrote recently, Twitter will win the fight for social brand advertising dollars.

De-pagination

After spending years and literally hundreds of blog posts about the proper way to paginate we’ll see a trend toward de-paginating in the SEO community. The change will be brought on by the advent of new interfaces and capabilities. (Blog post forthcoming.)

Analytics 3.0 Emerges

Pulling information out of big data will be a trend in 2013. But I’m even more intrigued by Google’s Universal Analytics and location analytics services like Placed. Marketers are soon going to have a far more complete picture of user behavior, Minority Report be damned!

Ingress Becomes Important

I’m a bit addicted to Ingress. At first you think this is just a clever way for Google to further increase their advantage on local mapping. And it is.

But XM is essentially a map Android usage. You see a some in houses, large clusters at transit stops, movie theaters and doctor’s offices, essentially anywhere there are lines. You also see it congregate at intersections and a smattering of it on highways.

Ingress shows our current usage patterns and gives Google more evidence that self-driving cars could increase Internet usage, which is Google’s primary goal these days.

Digital Content Monetization

For years we’ve been producing more and more digital content. Yet, we still only have a few scant ways to monetize all of it and they’re rather inefficient when you think about it. Someone (perhaps even me) will launch a new way to monetize digital content.

I Will Interview Matt Cutts

No, I don’t have this lined up. No, I’m not sure I’ll be able to swing it. No, I’m not sure the Google PR folks would even allow it. But … I have an idea. So stay tuned.

Have you noticed that direct traffic year over year is through the roof? Maybe you scratched your head, wrinkled your brow and chalked it up to better brand recognition. In reality, no such thing happened. What is happening is search traffic from iOS is being attributed to direct traffic instead.

Your organic search numbers are being mugged.

[Update] Frank Zimper notes that this problem also exists for those running Android 4.0 and higher. I’ve confirmed this via the same process you’ll read below. The only saving grace is that Android is usually a smaller traffic driver and the version migration is far more gradual. Yet, it’ll clearly continue to syphon search traffic off over time unless Google addresses this problem.

iOS 6 Search Theft

The reason these visits are being mis-attributed is a decision by Apple to move Safari search to secure (SSL) in iOS 6. The result of this decision is that the referrer isn’t passed. In the absence of a referrer Google Analytics defaults those visits to (none) which shows up in direct traffic.

The web browser on iOS 6 switched to use SSL by default and our web servers don’t yet take that fact into account. Searching still works fine, but in some situations the HTTP referer header isn’t passed on to the destination page. We’re investigating different options to address this issue.

As Google investigates different options to address this we’re left dealing with a serious data problem. Personally, I think Google Analytics should have a message within the interface that warns people of this issue until it’s fixed.

RKG did a nice job of tracking this and showing how to estimate the hidden search traffic. But for some reason this issue doesn’t seem to be getting as much traction as it should so I wanted to demonstrate the problem and show exactly how you can fight back. Because it’s tough enough being an SEO.

At a glance it looks like this has been a decent year for this client. But it’s actually better than it looks in October and November. Follow along to see just how much better.

Create iOS Advanced Segments

The first step is to create two Advanced Segments, one for iOS and one for iOS 6.

In May the labeling of Apple Operating Systems changed from specific devices to iOS. So include all four so you can see your iOS traffic for the entire year.

The iOS 6 segment is straightforward and will only be used to demonstrate and prove the problem. Also, if you want to perform this analysis on multiple analytics properties be sure to save these segments to any profile.

The Scene Of The Crime

Once you have your advanced segments you want to apply them as you look at direct traffic by month.

This plainly shows that direct traffic suddenly jumped from traditional levels upon the release of iOS 6 in late September.

Reclaiming Stolen Search Traffic

Every SEO should be reclaiming this stolen traffic to ensure they (and their clients) are seeing the real picture. Here’s my simple method of figuring out how much you should take back.

I’ve taken a three month slice of iOS traffic composed of April, May and June. From there I’m looking to see direct traffic as a percentage of the sum of direct and organic. The reason I’m not doing direct as a percentage of the total is to reduce any noise from referral spikes, paid search campaigns or other channel specific fluctuations.

In this instance direct comprises 10.5%. If you want to go the extra mile and quell the OCD demons in your head (or is that just me) you can do this for every month to ensure you’ve got the right percentage. I did and am confident that the percentage for this site is 10.5%.

Be aware, it will be different for each site.

Next I look at November and perform the same calculation just to confirm that it’s out of whack. At 46.6% it’s clearly departed from the established baseline.

I simply apply the proper direct traffic percentage (10.5% in this case) to the sum of direct and organic traffic. That’s the real amount of direct traffic. I then subtract that from the reported direct traffic to find the lost search traffic number.

The equation is none-((organic+none)*percentage). In this case I just reclaimed 79,080 search visits!

Better SEO Results

Get the credit you deserve and apply those stolen search visits to organic traffic.

A very quick calculation shows that reclaiming iOS search traffic produced a 4.6% bump in organic traffic for this client. That’s the best 32 minutes I’ve spent in a long time. Now it’s your turn.

TL;DR

Changes in how Safari searches are passed to Google Analytics is causing organic searches to be listed under direct traffic. Give clients the real picture and get the credit you deserve by properly attributing iOS traffic.

Twitter will steal Facebook’s bacon and become the most powerful brand advertising platform on the planet.

That’s saying a lot since I previously called Twitter the Underpants Gnomes of the Internet. But Twitter has changed and is no longer simply an altruistic agent of social change with revenue as a side gig. In 2013, Twitter means business.

That’s Entertainment

Those who have been on Twitter the longest probably still think of Twitter as an information source. You may remember back in 2009 when people began talking about how Twitter was their replacement for RSS feeds.

I was not one of those people. Don’t get me wrong, I found some value out of Twitter from an information perspective (and still do), but the signal to noise ratio was never that good.

But here’s what I’ve realized. Twitter is not about information anymore. It’s meta-entertainment.

Mark Cuban recently called Facebook a time waster, an alternative to boredom that looked far more like TV than a Google search. I think he’s right and his description applies to both Facebook and Twitter.

Supporting the idea of social media as entertainment is a March of 2012 The Hollywood Reporter study.

Nine of 10 respondents view social networking sites like Twitter and Facebook as a new form of entertainment, and more than half say social media sites are important tastemakers in determining what to watch and buy. Perhaps more surprising, 80 percent of television viewers visit Facebook while they watch.

Of course I have to believe these numbers might be a bit over-inflated based on who commissioned the study, but the general thesis resonates and seems solid.

Mobile Is Where It’s At

60% of our 140 million active users access Twitter through mobile – up from 55%. #Twitter4Brands

— Twitter Advertising (@TwitterAds) June 7, 2012

Twitter was mobile before it was sexy to be mobile. Mind you, it wasn’t really Twitter who figured out mobile. They had a robust community of third-party developers who led them to that conclusion over the course of many years. With all that data staring them in the face they moved quickly to double down on mobile. And it’s paid off.

Oh, did you notice the hashtag in that Tweet? Hello McFly!

Multi-Screen Viewing

Twitter’s lead in mobile has allowed them to capture the multi-screen viewing market. Make no mistake, this is the future of content consumption. Twitter understands that they can play a huge part, perhaps the connective tissue, between TV and other screens.

These are not this or that experiences but this and that experiences. Twitter is adding value to mass media content.

Pew found that 52% of adult cell phone owners use their phones while watching television. That’s the popular stat but it gets even more interesting if you look at just smartphone users.

Fully 74% of smartphone owners reported using their devices in one way or another while watching television in the preceding 30 days, compared with 27% of non-smartphone owners.

Of course, smartphones comprise the majority of phones (and rising), making this even more important. Yet, an April 2012 Forrester report shows that smartphones are already being displaced in many ways by tablets.

Tablets are displacing PCs and smartphones as the “couch computer” of choice: 85% of US tablet owners use their tablets while watching TV, and according to Nielsen, 30% of total tablet time is spent while watching TV.

The real takeaway here isn’t which screen is winning but that we’re entering a multi-screen viewing environment. Twitter, not Facebook, seems best positioned to capitalize on this new reality.

Owning The Hashtag

 

The hashtag is Twitter’s secret weapon.

While anyone can use a hashtag most consumers see it as synonymous with Twitter. I have to say I wasn’t a huge fan of the hashtag at first. Or, rather, I didn’t like the way many abused it, using more hashtags than normal words in a Tweet. (I still think that’s moronic.)

But hashtags are clearly a great way to aggregate content on a topic or event. Just watch a stream of Tweets from a conference and you’ll begin to understand the value of hashtags.

What’s more, when you’re attending a multi-track conference you often use the stream of Tweets from the sessions you don’t attend as a comparison and cheat sheet. It’s not unusual to hear someone complain that they were in the wrong session based on a comparison of Tweet streams.

Yet, I was still annoyed by hashtags until I read a piece by Denton Gentry on the use of hashtags to improve communication. Sure hashtags were great functionally but Denton made me realize that they were also ways to add expression.

Why does this matter in this context? The hashtag became entertainment. Hashtag memes were born and those brands who understood how to tap into this dynamic could create entertainment.

Collaborative Entertainment

The hashtag and Twitter’s short form anti-conversation content has created an opportunity for collaborative entertainment. It’s not about conversations it’s about the ever-changing aggregate of opinion, insight and snark.

I was recently down in Los Angeles on business and happened to be in the airport during the second Presidential debate. How did I wind up ‘watching’ it? On Twitter using the #debate stream. It was actually quite easy, interesting and fun to follow the debate this way.

I wasn’t going to wade into the mess of politics with my own Tweets but it was an entertaining way to view the debate and how others were interpreting it.

Pages vs Hashtags

Since the introduction of the Open Graph I really thought that Facebook wanted to kill Pages. In the last few years Facebook has made it more and more difficult for brands and businesses to make Pages worthwhile.

Yes, yes, I know you have a client or a case study that shows they’re killing it on Facebook but from the reduced functionality and reach I think most are swimming upstream on Pages.

I actually think it’s a smart idea to get rid of Pages but that’s a post for another day. The problem is Facebook has no alternative place to aggregate brand conversations. Unfortunately, Facebook doesn’t support the hashtag.

Facebook conversations are with brands. Twitter conversations are about brands.

This is really the functional difference between the two platforms right now. Many still cling to the notion that people want to have conversations with brands. I simply don’t think that’s true. Conversations with brands are not social. Yet that’s the implicit goal of Facebook Pages.

Conversely you need not follow a brand on Twitter to view that stream of hashtag content. I can tune in when I want and it doesn’t even need to be explicitly brand centric. Examples are littered over our television screens. Think of the hashtags on Survivor (#rewardchallenge and #immunitychallenge) or The Soup (#satanstoaster) to name just a few.

The hashtag is both a connection and platform for multi-screen collaborative entertainment.

User Centric Engagement

You don’t have to follow CBS or Survivor. You tune in when you want to tune in. Think about how scary and powerful that is!

The brand account could still be a valuable part of the ecosystem but it wouldn’t need to be the center of the brand experience. That might allow accounts to add value instead of incessantly trying to collect followers and figure out ways to break through the noise or crack the EdgeRank algorithm.

In fact, brands can participate in the hashtag stream along with everyone else, supplying ‘official’ announcements or insider content when appropriate. The role of an official account is to egg on your fans to provide that meta-entertainment.

Sure, the number of fans or followers seems comforting but we’ve all seen how little engagement results from these massive numbers. In the end it comes down not to who you follow but what content you’re engaging in.

Viewers Like You 

Imagine knowing which hashtag streams a user has viewed! How valuable would that information be? How easy would it be to advertise to lapsed viewers? Or to understand the other programming or products you might be losing out to based on viewing behavior. This isn’t about the brands I say I like but the ones I’m actively consuming.

The hand-wringing over active users as defined as those who Tweet or how many people they follow may be completely specious. The pure ‘lurker’ may be just as valuable, particularly for brand advertisers. I’d be far more interested to know about interaction based how many hashtag streams users viewed and the dwell time on those streams.

And there’s a really interesting opportunity to map hashtags to brands and categories, not to mention crawling the public social graph of accounts to develop demographic data. It would become relatively easy to match advertisers to users who frequently view a variety of hashtag streams.

The discussion around viewers makes me think about traditional TV advertising. Twitter seems to think so too if comments by Joel Lunenfeld at IAB MIXX are any indication.

A campaign on Twitter, he said, is “the ultimate complement to a TV buy.”

Can they make it any more clear?

Beyond Text

Twitter is doing a lot to make the experience more visual which is critical not just to keep up with competitors but to get mainstream adoption. And the new email Tweet feature continues to push them to a broader audience.

Again, I think Twitter is being relatively transparent in how they’re approaching this issue.

People tell incredible stories on Twitter through photos and videos. When you search for a person, an event or a hashtag, you can now see a grid of the most relevant media above the stream of Tweets.

You can also see media instantly in your search results stream on iPhone and Android. Photos and article summaries automatically show previews to give you a bird’s eye view on what’s happening.

This makes Twitter far more visual, compelling and … entertaining. The need for a consistent experience is also the reason why Twitter pulled back on the third-party apps and ecosystem as a whole.

You need a reliable, safe and consistent platform when securing major brand advertisers.

Context Matters

Facebook has a lot of advantages in being able to capture attention and profile interests. But there’s a fundamental problem with Facebook. It’s far more about navel gazing than anything else. The context is still largely personal.

Facebook aggregates your social graph while Twitter aggregates everything around a specific topic.

Even when someone shares something on Facebook it’s as much about who shared it with you as what is shared. You’re connected with the person not the content. Twitter is the other way around, with content coming first and people reduced to a filter.

Both platforms deliver a type of social voyeurism as entertainment, but the context is different.

Checking out the photos from a friend’s marathon run is not the most effective time or place to advertise running shoes. Sure the topic is right but the context is all wrong. I’m not looking at the marathon photos with shopping in mind. Heck I could hate running. Instead, I’m doing so because I want to keep up with my friend.

The person is important, the content isn’t. That’s not an optimal environment for advertising, even for intent generation.

Twitter Advertising

Twitter has been busy building out different advertising opportunities culminating recently in interest targeting. I’m not sure how this will all work for small businesses, but I don’t think anyone has fully solved that one yet.

However, I believe Twitter is laying the groundwork to catch traditional offline brand advertising dollars moving online. Twitter is creating a comfortable and recognizable entertainment platform that allows advertisers to connect and extend traditional channels.

Not only will brands and businesses want to advertise against these new forms of meta-entertainment, but they’ll seek out ways to create their own. There’s been a lot of talk about content marketing lately but what I see is the dawn of content advertising.

TL;DR

Twitter has quickly evolved into a collaborative entertainment platform that serves as the glue of multi-screen viewing. Their focus on mobile, visual makeover and tacit ownership of the hashtag puts Twitter and not Facebook in a position to capture the lion’s share of brand advertising dollars moving online.

That awkward moment when you realize you’ve been staring at interesting data for years without knowing it.

Every day you’re probably using Google Keyword Tool query volume in your SEO research. Of course you have to be careful to use the correct match type, right? You don’t want to make the mistake of promising broad match level volume to a client.

Recently I began to wonder about the differences in match type volume. Because they are substantial.

Keyword Match Ratio

What am I talking about? The keyword match ratio is the broad match volume of a keyword divided by the exact match volume of a keyword.

I know these are completely different keywords but the difference is pretty astounding. This metric should be meaningful. It’s not some end-all-to-be-all metric, but I believe the keyword match ratio is useful.

Here’s how I’ve been looking at and using the keyword match ratio.

Determining Intent

One of the main ways I’ve been using this new metric is in determining intent. Or, more specifically, is the intent uniform or fractured?

A low keyword match ratio indicates a more uniform syntax which often maps to uniform intent. In other words, there aren’t as many keyword variations of that term or topic. Uniform intent is great from a search perspective because you can more easily deliver a relevant and valuable experience for that traffic.

A high keyword match ratio indicates a less uniform syntax which may indicate fractured intent. That means there might be a lot of ways to talk about that topic or could point to a whole modifier class. Fractured intent is more difficult to satisfy since users may come with different expectations of value.

Unfortunately, determining intent got more difficult when Google reduced the level of category detail during the merge of Google Trends and Google Insights for Search.

You can still see that there’s potential fractured intent here but the old version would have presented the various percentage breakdowns for each category which was quite useful. Keyword match ratio provides a new way to validate whether you should be concerned about fractured intent.

Identifying Content Opportunities

The other way I’ve been using the keyword match ratio is to identify areas ripe for content creation. In this case, a high keyword match ratio indicates a potential for different modifiers and phrases for that keyword.

The term ‘hardwood floors’ has a pretty high keyword match ratio and even the suggested ad groups provide ample content ideas. Go a step further and use related searches and Google Autocomplete suggestions to get more ideas that match query syntax.

Look at all those content opportunities! Follow high keyword match ratios to uncover content ideas and opportunities.

Benchmarking

While I can usually just tell whether a keyword match ratio is high or low, or simply compare it to other keywords in a list, I wondered if I could create a benchmark. Enter Dr. Pete, who was kind enough to share the 1,000 keywords that comprise MozCast. (Thank you.)

The first thing I did was see how the keyword match ratio changed with query length.

As you might expect, the ratio declines as the number of words in the query increase. I like when things make sense! What this allows me to do is identify specific keywords that are materially outside of the norm.

What about the 2 word query with a ratio of 226.3 or the 2 word query with a ratio of 2.2. The ratio tells you something about the behavior of that keyword. It’s your job to figure out what it is.

Competition

My next idea was to map the ratio to keyword difficulty. I experimented with using the competition number via the Google Keyword Tool as a proxy but the numbers were all over the place.

So … I generated the keyword difficulty for 92% of the list five painstaking keywords at a time via the SEOmoz Keyword Difficulty Tool. (There’s a 300 a day limit so I didn’t quite get through the entire list.)

There might be a trend there but it was difficult to tell with all the noise. So I rounded keyword difficulty into deciles.

No terms fit into the 0, 10 or 100 deciles so I removed those rows from the table. What’s left does seem to indicate a rising keyword match ratio with increased keyword difficulty. That’s interesting and makes a bit of sense too. Competitive terms often have more volume and likely have a greater number of variants.

Putting It All Together

The question is how you can use all of this information together? To be honest, I haven’t come up with the perfect formula but I find it interesting to take terms and see where they fall against these benchmarks.

What about the term ‘swedish fish’? This 2 word keyword has a keyword match ratio of 3.3, well below the 2 word benchmark. In addition, with a 41% keyword difficulty it falls into the 40 bucket, which again puts it below the standard keyword match ratio for that difficulty.

That tells me the intent behind the term ‘swedish fish’ is uniform and it might be an area where a well optimized piece of content could rank well. Yum!

A term with a low keyword match ratio and low competition is a great SEO opportunity.

The syntax and intent are clear and you can provide relevant and useful content to fill that need. Of course, all of this has to produce productive traffic. We’re not doing SEO just for gold stars and pats on the back, right?

What about a term like ‘solar panels’? It has a keyword match ratio of 13.5, above the 2 word benchmark. With a keyword difficulty of 70% it also scores slightly over the average.

That tells me optimizing for ‘solar panels’ is going to be a hot mess. Instead, I’d want to look for phrases and modifiers that might be more attractive instead, with the long-term goal of building up to this head term.

Locate the specific intents and keywords that contribute to a high keyword match ratio and produce relevant content that satisfies and engages.

Context, Brains and Disclaimers

A couple of things you should know about the keyword match ratio. You need to use it in conjunction with other tools, in particular your brain. Context is important and different verticals and modifiers will have different keyword match ratio patterns.

So while I provide the benchmarks above you should be thinking about how the ratio fits into the keyword universe for your site, or for that particular modifier. If you were a coupon site you might want to see which store + coupons terms had the highest and lowest keyword match ratio.

There’s also the possibility that the set of data I used for the benchmark isn’t representative. However, I think Dr. Pete has done a pretty good job here and while some of the terms are strange and mundane that’s not a bad reflection of reality.

You’ll also note that I’m not doing any heavy duty statistical analysis here. While I understand and enjoy those endeavors I think pattern recognition can take you pretty far pretty quickly. Maybe someone else can pick up this thread and create something more statistically valid.

In the interim, I’m using the keyword match ratio as an SEO hack to help me find potential diamonds in the rough and areas for content creation.

TL;DR

The keyword match ratio measures the ratio of broad match volume and exact match volume. This metric is not fool proof. You need to use your brain when looking at it. But if you’ve got a good head on your shoulders the keyword match ratio can help you determine intent and sniff out content opportunities.