You Are Browsing The Analytics Category

Image Blind

December 16 2014 // Analytics + SEO // 6 Comments

Images are an increasingly important part of the Internet landscape. Yet marketers are provided very little in the way of reliable metrics to allow us to understand their power and optimize accordingly. This is doubly strange given the huge amount of research going on regarding images within search engine giants such as Google.

Image Tracking In Google Analytics

There is none. Or at least there is no image search tracking in Google Analytics unless you create filters based on referrers. I wrote about how to track image search in Google Analytics in March of 2013 and updated that post in April of 2014.

The problem with this method is that it is decreasing in usefulness. I still use it and recommend it because some visibility is better than none. But when Chrome removed the referrer completely from these clicks earlier this year it really hurt the accuracy of the filter.

Who cares you might be asking. I care because image search intent and the resulting user behavior is often wildly different than web search.

Google Image Search Traffic Behavior

The users coming to the site above via web search have vastly different behavior metrics than those coming from image search. I’ve highlighted the dramatic pages per visit and time on site metrics. Shouldn’t we be building user stories and personas round this type of user?

For a while I explained away the reasons for not providing image search tracking in Google Analytics under the umbrella of privacy. I understand that Google was pretty much forced to move to ‘not provided’ because of lawsuits, Gaos v. Google Inc. in particular. I get it.

But I’m with Chris Messina. Privacy shouldn’t be a four letter word. And the one company who has the best chance of changing the conversation about it is Google. But let’s not go down the privacy rabbit hole. Because we don’t have to.

Right now Google Analytics provides other data on how people search. They break things down by mobile or tablet. We can even get down to the device level.

Google Analytics by Device

Are we really saying that knowing the user came in via image search is more identifiable than what device they were using? They simply explain different meta data on how a user searched.

Furthermore, on both web and image search I can still drill down and see what page they landed on. In both instances I can make some inferences on what term was used to get them to that page.

There is no inherent additional data being revealed by providing image search as a source.

Image Clicks in Google Webmaster Tools

I wouldn’t be as frothed up about this if it was just Google Analytics. Because I actually like Google Analytics a lot and like the people behind it even more.

But then we’ve got to deal with Google Webmaster Tools data on top and that’s an even bigger mess. First let’s talk about the dark pattern where when you look at your search queries data it automatically applies the Web filter. #notcool

Default Web Filter for Search Queries in GWT

I’m sure there’s an argument that it’s prominent enough and might even draw the user’s attention. I could be persuaded. But defaults are dangerous. I’d hazard there are plenty of folks who don’t even know that you can see this data with other filters.

And a funny thing happens with sites that have a lot of images (think eCommerce) when you look at this data. It doesn’t make an ounce of sense.

What happens if I take a month’s worth of image filtered data and a month’s worth of web filtered data and then compare that to the actual data reported in Google Analytics?

Here’s the web filtered data which is actually from November 16 to December 14. It shows 369,661 Clicks.

GWT Web Filter Example

Now here the image filtered data from the same time frame. It shows 965,455 Clicks.

GWT Image Filter Traffic Graph

Now here’s what Google Analytics reports for the same timeframe.

Google Analytics Traffic Comparison

For those of you slow on the uptake, the image click data from Google Webmaster Tools is more than the entire organic search reported! Not just Google but organic search in total. Put web and image together and we’re looking at 1.3 million according to Google Webmaster Tools.

I’m not even going to get into the ratio of image clicks versus web clicks and how they don’t have any connection to reality when looking at the ratio in Google Analytics. Even taking the inaccuracy of the Google Analytics filters into account it points to one very clear truth.

The image click data in Google Webmaster Tools is wonky.

So that begs the question. What exactly is an image click? It doesn’t seem to be limited to clicks from image search to that domain. So what does it include?

This blog is currently number three for the term ‘cosmic cat’ in image search (#proud) so I’ll use that as an example.

What Is an Image Click?

Do image clicks include clicks directly to the image, which are generally not on that domain and not counted in most traffic packages including Google Analytics? Maybe. But that would mean a lot of people were clicking on a fairly small button. Not impossible but I’d put it in the improbable bucket.

Or do image clicks include any time a user clicks to expand that image result? This makes more sense given what I’m seeing.

But that’s lunacy. That’s comparing apples to oranges. How does that help a marketer? How can we trust the data in Google Webmaster Tools when we encounter such inconsistencies.

Every webmaster should be inquiring about the definition of an image click.

The definition (of sorts) provided by Google in their support documentation doesn’t help.

GWT Search Queries FAQ

The first line is incorrect and reflects that this document hasn’t been updated for some time. (You know, I hear care and attention to detail might be a quality signal these days.) There’s a line under devices that might explain the image click bloat but it’s not contained in that section and instead is attributed to devices.

Long story short, the documentation Google Webmaster Tools provides on this point isn’t helpful. (As an aside, I’d be very interested in hearing from others who have made the comparison of image filter and web filter clicks to Google Analytics traffic.)

Images During HTTPS Conversion

These problems came to a head during a recent HTTP to HTTPS conversion. Soon after the conversion the client involved saw a decent decline in search traffic. Alarm bells went off and we all scrambled to figure out what was going on.

This particular client has a material amount of images so I took the chart data from both HTTP and HTTPS for web and image clicks and graphed them together.

Exasperated Picard

In doing so the culprit in the decline post conversion was clearly image traffic! Now, some of you might be thinking that this shows how the Google Webmaster Tools data is just fine. You’re be wrong! The data there is still incorrect. It’s just wrong consistently enough for me to track fluctuations. I’m glad I can do it but relying on consistently bad data isn’t something I’m cheering about.

The conclusion here seems to be that it takes a long time to identify HTTPS images and match them to their new HTTPS pages. We’re seeing traffic starting to return but it’s slower than anyone would like. If Google wants sites to convert to HTTPS (which they do) then fixing this image search bottleneck should be a priority.

Image Blind?

I'm Mad as Hell And ...

The real problem here is that I was blindsided due to my lack of visibility into image search. Figuring out what was going on took a fair amount of man hours because the metrics that would have told us what was going on weren’t readily available.

Yet in another part of the Googleplex they’re spending crazy amounts of time on image research.

Google Image Advancements

I mean, holy smokes Batman, that’s some seriously cool work going on. But then I can’t tell image search traffic from web search traffic in Google Analytics and the Google Webmaster Tools data often shows more ‘image clicks’ to a site than total organic traffic to the site in the same time period. #wtf

Even as Google is appropriately moving towards the viewable impressions metric for advertisers (pdf), we marketers can’t make heads or tails of images, one of the most important elements on the web. This needs to change.

Marketers need data that they can both rely on and trust in to make fact based decisions.

TL;DR

Great research is being done by Google on images but they are failing marketers when it comes to image search metrics. The complete lack of visibility in Google Analytics coupled with ill defined image click data in Google Webmaster Tools leaves marketers in the dark for an increasingly important type of Internet content.

Twitter Analytics

August 11 2014 // Analytics + Social Media // 21 Comments

What if Twitter launched the most awesome analytics dashboard and no one really noticed? Well, that’s pretty much what happened nearly a month ago. I’ve been waiting for the posts that detail how much you can get from the tool and the different types of analysis you can perform.

But … I’m tired of waiting.

Twitter Analytics Dashboard

The dashboard provides a decent overview of activity over the last 28 days.

Twitter Analytics Dashboard Overview

The major statistics it provides are Impressions, Engagements and Engagement Rate for each tweet and the trend for those over time. That’s not too shabby but lets poke at what lurks under Engagements.

Twitter Engagements

Click on a specific Tweet and you get to see how people engaged with that specific Tweet.

Twitter Analytics Tweet Engagements

Now if you’re not quietly swearing under your breath at this point I don’t know what’s wrong with you. There’s so much awesome information here. A sliding scale of engagement for you to pour over.

In particular, you can see which Tweets produced User profile clicks and actual Follows. Not shown here but also tracked are the number of times the Tweet was Shared via email. But wait, we haven’t even gotten to the best part.

Export And Analyze

Twitter Analytics Export Data Button

At the top right hand on the dashboard is an Export data button. This might as well be colored gold and in the shape of a treasure chest. Click and suddenly you have one of the richest sets of data you could wish for on your Tweets.

Twitter Analytics Export Data in Excel

This eyesore of data is a goldmine. You get the actual text of each Tweet along with the timestamp coupled with all of the engagement metrics. So what could you learn from this data?

A bit of data manipulation and I can find out which days I have the most engagement.

Tweets by Day of Week Chart

Monday and Thursday for the small amount of time I have data. But maybe I just want to see the overall engagement rate by day.

Twitter Analytics Engagement Rate by Day of Week Chart

Friday and Saturday suddenly look pretty good from an engagement efficiency standpoint. I could drill down here and get to the hour and come out with one of those popular ‘best time to Tweet’ posts if I wanted. But I won’t.

Twitter Analysis Smorgasbord

Instead I’ll look for better insights. I happen to use hashtags as a way to classify my Tweets. Two of the more popular ones I use are #seo and #ux. Now with a bit more data manipulation I can look at how these two different themes of Tweets perform.

Twitter Analytics Engagement by Hashtag

I get a lot more impressions and engagements overall with the #seo hashtag but my engagement rate on #ux is twice as high. I could dig even deeper and do a pivot table to see what type of engagement I’m getting on each.

Twitter Analytics Types of Engagement by Hashtag

It’s hard to see, I know, but here I can tell that I get more retweets per Tweet on #seo but that many of the other metrics skew towards #ux in terms of engagement efficiency. This makes sense to me since I’m more of an authority in SEO than in UX. But it shows that with the right type of Tweets I am moving the needle in the latter. (Engagement efficiency – that has a nice ring to it doesn’t it?)

The analytic opportunities here are nearly endless. Particularly if you’ve adhered to some sort of pattern in your Tweets (thank you latent OCD).

Twitter Analytics Engagement by Prefix Graph

So here I can see that my particular Tweet pattern of using a prefix gives me some interesting results. Do people pay more attention and interact with my Tweets when I say I’m saving the piece of content I’m referencing? Maybe. But there’s also a huge bias involved in the value of that content. Either way it’s something I can track over time.

So what are you waiting for?

How To Get Twitter Analytics

I think part of the problem is that the analytics feature is buried under the Ads interface. Maybe folks think you need to be running ads to get all of the organic Tweet data. That’s not true. I haven’t been running ads on my account. Never have. All I did was click the Get Started link and jump through a few hoops. Free!

If you’re having trouble check out Dan Shure’s post about how to set up Twitter Analytics on Evolving SEO.

Flabbergasted LOLcat

Hopefully you’re ready to jump in with both feet and try this out. I know i’d appreciate others providing some insight and potentially some macros to make the analysis even easier. Step to it Excel gurus!

[Updated August 22nd, 2014] Dan Shure at Evolving SEO also has some tips on using Favorited Rate to predict content success.

[Updated September 24th, 2014] Paul Shapiro at Search Wilderness also pointed me at Twitter Analytics for Websites. I implemented this a month ago and have had it in a Chrome tab ever since. What’s cool is that it gives you information about everyone Tweeting about your website.

Twitter Analytics for Websites

So implement both to gain insight into how both you and your site is performing.

TL;DR

Twitter is giving you an amazing dashboard and data on your organic Tweets that allows you to perform an insane amount of powerful social analysis.

Tracking Image Search In Google Analytics

March 27 2013 // Analytics + SEO // 56 Comments

(This post has been updated as of 4/5/14 to reflect refinements to the filters as well as new caveats about Chrome.)

The Internet is becoming increasingly visual but the standard Google Analytics default lumps image search traffic in with organic traffic. The problem with that is these two types of traffic have radically different behaviors.

Google Analytics Y U No Track Image Search

So here’s a quick way for you to track image search in Google Analytics to gain insight into how images are performing for your business.

Image Search Referrers

After the last big image search update I was asked by Annie Cushing if I’d figured out a way to track images in Google Analytics. I’d meant to but hadn’t yet. Her reminder led me to find out what was possible. I fired up Firefox and used Live HTTP Headers to look at the referrers for image search traffic.

I found that there were two distinct referrers for Google, one from Google images and one from images that showed up via universal search results.

Here’s what the referrer looks like from Google image search.

Google Image Search Referrer

The parts to note here are the /url? and the source=images parameter. Now lets look at what the referrer looks like from an image via universal search.

Google Image Referrer via Universal Search

The part to note here is that the URL doesn’t use /url? but imgres? instead. This means you can track traffic from each source!

But there’s another wrinkle I discovered over time. Many of the international versions of Google use the old image search UX which also produces the /imgres? referrer.

google.fr image search for ruby red slippers

In addition, most of these wind up being passed in the Google cookie as a ‘referring’ medium and not ‘organic’. So you might be seeing Google domains cropping up in your referring reports (annoying!). Adding Full Referrer as a secondary dimension shows where the majority of these are coming from: imgres.

Google Referring Traffic in Google Analytics Reports

This means two things. First, we’re going to have to create a special case for universal search on google.com so that it isn’t mixed up with image search from international properties. Second, we’re going to have to change the medium on the international image search traffic so that it is properly attributed to organic.

Finally lets take a look at Bing.

Bing Image Search Referrer

This is pretty straight forward and doesn’t change based on whether it’s from image search proper or via a universal result.

Google Analytics Image Search Filters

If you know the referrer patterns you can set up some Google Analytics filters to capture and reclassify this traffic into the appropriate buckets. Here’s the step-by-step way to do that.

From Google Analytics click Admin.

Google Analytics Admin

That takes you to a list of profiles.

Google Analytics Select or Create a Profile

Here you can either create a new profile or select a current one. I’d suggest creating a new profile to test this out before you decide to integrate it into your primary profile. Because you might screw it up or just may not like the detail or may not want to have the change in continuity. That said, I’ve created these filters so they’ll have the least amount of impact on your reporting while still delivering added insight.

Next you’ll reach the profile navigation pane where you’ll want to click on Filters.

Google Analytics Filters 2014

At that point you’ll want to go ahead and click the red New Filter button.

Google Analytics Red New Filter Button

That’s when the real fun begins and you construct a new advanced filter.

Creating a Google Analytics Google Image Search Filter

The first step is to name this filter. This won’t show up in your reports and is simply a way for you to know what that filter is doing. So make it descriptive and obvious.

Next you’ll want to select the Custom filter button (2) which then reveals a list of options. From that list you’ll want to select Advanced (3). This is where it gets a bit tricky.

In step 4 you’ll select Referral from the menu of options and then apply some RegEx to match the pattern we’ve identified. In this instance the RegEx I’m using is:

.*google\.(.*)/url.*source=images.*

I love RegEx, which stands for Regular Expression, but I don’t always get it right the first time and regularly rely on this RegEx cheat sheet to remind and guide me. In this instance I’m looking for all Google domains (and  including any international domain using the new image search here) with /url and source=images within the referrer.

In step five you’re selecting what you’re going to do when a referrer matches your RegEx. I’ve chosen Campaign Source from the menu and then created a new source called ‘google images’. You can name these whatever you like but I keep them lowercase to match the other sources.

You’ll note that the ‘Override Output Field’ is set to Yes which means that I’m going to change the Campaign Source for those that match this referrer pattern from what it is currently to ‘google images’. The great part about this is that you retain the fact that the medium is ‘organic’. So all those reports remain completely valid.

Finally, you click Save and then you wait for the filter to be applied to traffic coming into the site. Depending on the amount of traffic you get from these sources, it may take a few hours to a few days to see the filter working in your reports.

Next we have to put into place a filter for Google universal images, Google images from international properties not using the current image search UX as well as Bing images.

The RegEx for Google universal search images is:

.*google.com/imgres.*

Note that I’m only looking to match referrers coming from google.com so that I’m not mixing international image search with US universal image search.

The RegEx for Google international search is crazy long and didn’t really work pasted here. So instead you can click here to copy and paste the Google ‘International’ image search filter RegEx.

Now, many of the domains won’t match because they’re using the new version of image search, which will match the first filter we created. But I figured I’d just be as inclusive as possible instead of validating the current image search UX on each domain. (I mean, it’s wicked time consuming too.)

Finally, the RegEx for Bing images is:

.*bing\.(.*)/images/search.*

But we’re not done! Close, but not quite.

Changing Google Analytics Medium Filters

So after having these filters in place for a while I noticed that some of the new sources I created were showing up as a medium of ‘referring’ instead of ‘organic. That means you’re still short-changing your organic efforts because Google is passing the wrong medium in their cookie.

So you have to create two new filters that change the medium of Google universal images and Google international images.

Google Analytics Filter to Change Medium

This is another Advanced filter but this one is much simpler but must be very precise. In Field A  you’re looking for the Campaign Source that exactly matches the source you created in the filter. For me, that means ‘google international images’ and ‘google universal images’. For you, it’s whatever you named the new sources.

Then you’re simply outputting and overriding the Campaign Medium to organic. Remember, you’ll create two of these. One for the ‘international’ images and one for ‘universal images’. My guess is that you might only need the one but I want to cover my bases.

To simplify, all your doing here is looking for the sources you created and then making sure that the medium associated with those sources is changed to organic.

Image Search Filter Order

The final step is to make sure that your filters are in the right order. The last two filters that change the medium based on a specific campaign source (that you created) must come at the end.

Google Analytics Google Image Search Filter Order

This makes sense right? You couldn’t match a source that you hadn’t already created, right? Stick to this order and you’ll ensure image search traffic is tracked appropriately.

Image Search Reports

So what do you get to see in the reports?

Image Filters Create Better Google Analytics Reports

This is data from a client site where I’ve had all the filters in place for a few days. The medium for all of these is still organic but I’ve now got new sources for google images, universal images and bing images. (Update on 4/5/14) I’ve been using these filters successfully for a year now.

What you should see right away is the very large difference in how this traffic performs. Image search traffic in this instance has a 1.5 Pages/Visit and 3:00 Avg. Visit Duration while the web based organic traffic has a 6 Pages/Visit and 6.00 Avg. Visit Duration.

Most importantly, the conversion rate on these two types of traffic is different as well. Segmenting your image search traffic can bring more clarity to your analysis and help you make the right decisions on what’s working, how to allocate resources and what to optimize.

Image Search Filter Validation

So how do I know this is really working? I drill down into one of these new sources and then select keyword as the secondary dimension. Did I forget to mention that the keyword data remains in tact?

Google Analytics Universal Images Keyword Report

Yup, sure does! So the next step here is to see if there really is a universal result for these keywords.

Google Search Result for Badass Over Here Real Pic

Sure enough, I’m the second result in this universal search result. Now lets see if the filter for normal image search is working.

Google Analytics Google Images Keyword Report

I’ll use ‘wifi logo’ as my target term and first go to make sure that I’m not showing up in universal search results.

Google Search Result for Wifi Logo

Nope, not showing up there. But am I showing up in Google image search?

Google Images Search Results for Wifi Logo

Sure enough I’m there just inside the top 100 results from what I can tell. So I’m pretty confident that the filter is catching things and bucketing them appropriately. I’ve also validated this with very robust client data but can’t share that level of detail publicly.

What Is images.google?

You might have noticed the images.google source above. What’s that you ask? I don’t know. But I don’t think it’s traditional image search traffic since the user behavior of that source doesn’t conform to the other three image based sources. It’s also a small source of traffic so while my OCD senses are tingling I’m currently ignoring the urge to figure out exactly what images.google represents.

Tell me if you figure it out.

Caveats

You Raise a Valid Point Ice Cream

The big question is why I wouldn’t just use the Google Webmaster Tools queries report and filter by image right? Well first off, the integration into Google Analytics still isn’t where I’d like it to be making any type of robust reporting near impossible.

In addition, I don’t like mixing image search traffic with web search traffic in my normal reports because they’re so different. It makes any analysis you do using that mixed data less precise and prone to unintentional error.

More problematic is the fact that the data between Google Webmaster Tools and Google Analytics doesn’t match up.

I started looking at specific keywords via my filters versus what was reported in Google Webmaster Tools. There were just too many times when Google Webmaster Tools reported material amounts of traffic that wasn’t showing up in my Google Analytics reports.

Google Webmaster Tools Clicks

Here you can see that the top term received 170 clicks in this time frame. Yet during the same time frame here’s what the Google Analytics filter based method reports.

Google Analytics Image Based Clicks

170 versus 24! Even if I factor in the (not provided) percentage (which runs about 35% for this client) and add that back in I only get close to 40 visits.

But that’s when the lightbulb went off. Maybe Google Analytics is reporting Visits while Google Webmaster Tools is reporting Clicks?

While I can’t confirm this I’m guessing that Google Webmaster Tools is counting all clicks on a result. Many of those clicks are going directly to the image and not the page the image resides on. That’s important since direct clicks to the image (i.e. – .jpg files and the like) aren’t going to be tracked in Google Analytics as a visit. There is no Google Analytics code on these files. The delta between the two could be the number of users who clicked directly to the image.

In addition, this method doesn’t catch any of the mobile clicks and visits since no image search visits (and very few universal images) show up using this filter when looking at mobile traffic. I’m pretty sure that the referrers are just getting stripped and these wind up going into direct instead which is part of the iOS and Android 4+ search attribution issue. (If someone else has an explanation here or finds a different referrer for mobile image search please let me know.)

Finally, there’s something funky with Chrome. When I look at the distribution of traffic to each bucket Chrome is an outlier for Google images.

Image Filters Browser Distribution

That 3.7% is just way out of proportion. And it’s not related to the amount of (not provided) traffic since Firefox actually has a higher percentage of (not provided) 72% than Chrome (64%) in this instance. So I can only conclude that there’s some amount of data loss going on with Chrome. Maybe that also contributes to the discrepancy I see between Google Analytics and Google Webmaster Tools.

This got even worse as of January when Chrome stopped passing rich referrer information.

Image Search by Browser

I can only guess that this is part of Google’s security and privacy efforts. Sadly, it means you’re capturing a lot less detail about image search and your data will be less accurate because of it.

Despite all of these caveats I love having the additional detail on image traffic which has wildly different intent and user behavior. Some insight is better than none.

TL;DR

Apply a few simple Google Analytics filters to gain insight into how much traffic you’re getting through image search. This is increasingly important as the Internet becomes more visual and the user behavior of these visits differs in material ways from traditional search traffic.

New Ways To Track Keyword Rank

January 13 2013 // Analytics + SEO // 83 Comments

Tracking keyword rank is as old as the SEO industry itself. But how you do (and use) it is changing. Are you keeping up?

This post covers how I create and use rank indexes and introduces a new and improved way to track rank in Google Analytics.

Rankaggedon

In December of 2012 both Raven and Ahrefs made the decision to shut down their rank tracking features because they violated Google’s Terms of Service. The reaction from the SEO industry was predictable.

WTF LOLcat

The debate about why Google began to enforce the TOS (I think it has to do with the FTC investigation) and the moaning about how unfair it is doesn’t interest me. Both SEOmoz and Authority Labs still offer this service and the way many use rank needs to change anyway.

Every obstacle is an opportunity. Trite but true.

Is Rank Important?

To be honest, I don’t use rank that much in my work. This has to do with a combination of the clients I choose to work with and my philosophy that increasing productive traffic is the true goal.

Yet, you’d have to be soft in the head not to understand that securing a higher rank does produce more traffic. Being on the first page matters. Getting in the top three results can produce significant traffic. Securing the first position is often a huge boon to a business. Duh!

But rank is the extrinsic measurement of your activities. It’s a Google grade. Rank isn’t the goal but the result.

Unfortunately, too many get obsessed with rank for a specific keyword and spend way too much time trying to move it just one position up by any means necessary. They want to figure out what the teacher is going to ask instead of just knowing the material cold.

Rank Indexes

So how do I use rank? I create rank indexes.

A rank index is the aggregate rank of a basket of keywords that represent a type of query class that have an impact on your bottom line. For an eCommerce client you might have a rank index for products and for categories. I often create a rank index for each modifier class I identify for a client.

Usually a rank index will contain between 100 and 200 keywords that represent that query class. The goal is to ensure that those keywords reflect the general movement of that class and that changes in rank overall will translate into productive traffic. There’s no sense in measuring something that doesn’t move your business.

If that rank index moves down (lower is better) then you know your efforts are making a difference.

Executives Love Indexes

Business Cat

A rank index is also a great way to report to C Level executives. These folks understand index funds from an investment perspective. They get this approach and you can steer them away from peppering you with ‘I did this search today and we’re number 4 and I want to be number 1′ emails.

It becomes not about any one term but the aggregate rank of that index. That’s a better conversation to have in my opinion. A rank index keeps the conversation on how to move the business forward instead of moving a specific keyword up. 

Getting Rank Index Data

If you’re using SEOmoz you export the entire keyword ranking history to CSV.

SEOmoz Export Full Keyword History to CSV

After a bit of easy clean up you should have something that looks like this in Excel.

SEOmoz Keyword History Raw Data

At this point I simply copy and paste this data into my prior framework. I’ve already configured the data ranges in that framework to be inclusive (i.e. – 50,000 rows) so I know that I can just refresh my pivot table and everything else will automagically update.

If you’re using Authority Labs you’ll want to export a specific date and simply perform the export each week.

Authority Labs Keyword Ranking Export

There’s a bit more clean up for Authority Labs data but in no time you get a clean four column list.

Authority Labs Keyword Data

Unlike the SEOmoz data where you replace the entire data in your framework, you simply append this to the bottom of your data. Once again, you know the pivot table will update because the data range has been configured to be quite large.

Creating The Rank Index Pivot Table

You can review my blow by blow of how to create a pivot table (though I’m not using a new version of Excel so it all looks different anyway.) It’s actually a lot easier now than it was previously which is something of a miracle for Microsoft in my view.

Keyword Rank Index Pivot Table

You’ll use the keyword as your row label, date as the column label and the Average of rank as the values. It’s important to use a label so you can create different indexes for different query classes. Even if you only have one index, use a label so you can use it as a filter and get rid of the pesky blank column created by the empty cells in your data range.

You may notice that there are a lot of 100s and that is by design.

Keyword Rank Index Pivot Table Options

All those non-ranked terms need to be counted somehow right? I chose to use 100 because it was easy and because Authority Labs reports up to (and sometimes beyond) that number.

Turning Rank Data Into A Rank Index

Now that you have all the rank data it’s time to create the rank index and associated metrics.

Keyword Rank Index Calculated Data

Below the pivot table it’s easy to use a simple AVERAGE function as well as various COUNTIF functions to create these data points. Then you can create pretty dashboard reports.

Keyword Rank Index Reports

Average Rank is the one I usually focus on but the others are sometimes useful as well and certainly help clients better understand the situation. A small caveat about the Average Rank. Because you’re tracking non-ranking terms and assigning them a high rank (100) the average rank looks a bit goofy and the movement within that graph can sometimes be quite small. Because of this you may wind up using the Average of Ranking Terms as your presentation graph.

Average of Ranking Terms Graph

I don’t care much about any individual term as long as the index itself is going in the right direction.

Projecting Traffic

I can always look at the details if I want and I’ve also created a separate tab which includes the expected traffic based on the query volume and rank for each term.

Rank Index Traffic Projections

This simply requires you to capture the keyword volume (via Google Adwords), use a click distribution table of your choosing and then do a VLOOKUP.

IFERROR(([Google Adwords Keyword Volume])*(VLOOKUP([Weekly Rank],[SERP Click Distribution Table]),2,0)),0)

You’ll need to divide by 4 to get the weekly volume but at that point you can match that up to real traffic in Google Analytics by creating a regex based advanced segment using the keywords in that index.

Of course, you have to adjust for (not provided) and the iOS attribution issue so this is very far from perfect. And that’s what got me really thinking about whether rank and rank indexes could be relied on as a stable indicator.

What is Rank?

What Is Love Night at the Roxbury

The rise in (not provided) and the discrepancies often seen between reported rank volume and the traffic that shows up point to the increase in personalization. SERPs are no longer as uniform as they once were and personalization is only going to increase over time.

So you might have a ‘neutral’ rank of 2 but your ‘real’ rank (including context and personalization) might be more like a 4 or 5.

That’s why Google Analytics rank tracking seems so attractive, because you can get real world ranking data based on user visits. But that method is limited and makes reporting a huge pain in the ass. The data is there but you can’t easily turn it into information … until now.

Improved Google Analytics Rank Tracking

I got to talking to Justin Cutroni (a really nice and smart guy) about the difficulties around tracking rank in Google Analytics. I showed him how I use rank indexes to better manage SEO efforts and over the course of a conversation (and a number of QA iterations) he figured out a way to deliver keyword rank the way I wanted in Google Analytics.

Keyword Rank Tracking In Google Analytics with Events

Using Events and the value attached to it, we’ve been able to create real keyword rank tracking in Google Analytics.

The Avg. Value is calculated by dividing the Event Value by Total Events. You could change this calculation once you do the export to be Event Value by Unique Events if you’re concerned about those users who might refresh the landing page and trigger another Event. I haven’t deployed this on a large site yet to know whether this is a real concern or not. Even if it is, you can always change it in the export.

Keyword Rank Tracking Data via Analytics Events

So you can just make Avg. Value a calculated field and then continue to tweak the exported data so that it’s in a pivot table friendly format. That means adding a date column, retaining the Event Action column but renaming it keyword, adding a Tag column, and retaining the Avg. Value column.

You essentially want it to mimic the four column exports from other providers. I suppose you could keep a bunch of this stuff in there and not use it in the pivot table too. I just like it to be clean.

Event Based Rank Tracking Code

Start tracking rank this way on any Google Analytics enabled site by dropping the following code into your header.

Google Analytics Rank Tracking Code

To make it easier, the code can be found and copied at jsFiddle. Get it now!

Just like the old method of tracking rank in Google Analytics, this method relies on finding the cd parameter (which is the actual rank of that clicked result) in the referring URL. This time we’re using Event Tracking to record rank and putting it in a field which treats it as a value.

The code has also been written in a way to ensure it does not impact your bounce rate. So there’s no downside to implementation. You will find the data under the Content > Events section of Google Analytics.

Where To Find Average Rank in Google Analytics

Just click on Content, Top Events and then RankTracker and you’ll find keyword ranking data ready for your review.

Google Analytics Rank Indexes

I’ve been working at applying my index approach using this new Event based Google Analytics rank tracking data. The first thing you’ll need to do is create an advanced segment for each index. You do this by creating a regex of the keywords in that index.

Rank Index Regex Advanced Segement

Sometimes you might not get a click on a term that is ranked 20th and certainly not those that are ranked 50th. That’s a constraint of this method but you can still populate an entire list of keywords in that index by doing a simple VLOOKUP.

IFERROR(VLOOKUP(A1,'Export Event Data'!$A$1:$E$5000,5,FALSE),100)

The idea is to find the keyword in your export data and report the rank for that keyword. If the keyword isn’t found, return a value of 100 (or any value you choose). From there it’s just about configuring the data so you can create the pivot table and downstream reports.

Caveats

You Raise a Valid Point Ice Cream

This new way of tracking is different and has some limitations. So lets deal with those head on instead of creating a grumble-fest.

The coverage isn’t as high as I’d like because of (not provided) and the fact that the cd parameter is still only delivered in about half of the referrers from Google. I’m trying to find out why this is the case and hope that Google decides to deliver the cd parameter in all referrers.

Full coverage would certainly increase the adoption of rank tracking in Google Analytics and reduce those seeking third party scraped solutions, something Google really doesn’t like. It’s in their self-interest to increase the cd parameter coverage.

As an aside, you can get some insight into the rank of (not provided) terms and match those to landing pages, which could be pretty useful.

Rank of Not Provided Terms by Landing Page

The other limitation is that you only get the rank for those queries that received clicks. So if you’re building a rank index of terms you want to rank for but aren’t and track it over time it becomes slightly less useful. Though as I’ve shown above you can track the average of ranking terms and of the index as a whole at the same time.

One of the better techniques is to find terms that rank at 11 to 13 and push them up to the front page, usually with some simple on-page optimization. (Yes, seriously, it’s way more effective than you read about.) So this type of tracking might miss a few of these since few people get to page 2 of results. Then again, if you see a rank of 11 for a term with this tracking that’s an even higher signal that getting that content to the front page could be valuable.

Finally, the data configuration is, admittedly, a bit more difficult so you’re working a tad harder to get this data. But on the other hand you’re seeing ranking data from real users. This could get really interesting as you apply geographic based advanced segments. Larger organizations with multiple locations might be able to determine which geographies they rank well in versus those where they’re struggling.

And not Or

At this point I can’t say that I’d scrap traditional rank tracking techniques altogether, though I’m sure Google would like me to say as much. Instead, I think you should use the new Google Analytics Event Based Rank Tracking in conjunction with other ranking tools.

First off, it’s free. So there’s no reason not to start using it. Second, you get to see real world rank, which while limited in scope can be used to compare against neutral rank offerings. Lastly, if you’re trying to future proof your efforts you need to be prepared for the potential end to traditional ranking tools or such high variation in personalization to make them unreliable.

Did I mention this new rank tracking method is free?

I’m looking forward to putting this into practice and comparing one tracking method to the other. Then we’ll see the potential variance between personalized ranking versus anonymized ranking.

TL;DR

The closure of recent third-party rank tracking services is an opportunity to think about rank in a different way. Using a rank index can help keep you focused on moving the business forward instead of a specific keyword. To future proof your efforts you should implement improved Google Analytics rank tracking for free.

Reclaiming Lost iOS Search Traffic

December 19 2012 // Analytics + SEO // 29 Comments

Have you noticed that direct traffic year over year is through the roof? Maybe you scratched your head, wrinkled your brow and chalked it up to better brand recognition. In reality, no such thing happened. What is happening is search traffic from iOS is being attributed to direct traffic instead.

Your organic search numbers are being mugged.

[Update] Frank Zimper notes that this problem also exists for those running Android 4.0 and higher. I’ve confirmed this via the same process you’ll read below. The only saving grace is that Android is usually a smaller traffic driver and the version migration is far more gradual. Yet, it’ll clearly continue to syphon search traffic off over time unless Google addresses this problem.

iOS 6 Search Theft

Stolen Search Traffic LOLcat

The reason these visits are being mis-attributed is a decision by Apple to move Safari search to secure (SSL) in iOS 6. The result of this decision is that the referrer isn’t passed. In the absence of a referrer Google Analytics defaults those visits to (none) which shows up in direct traffic.

The web browser on iOS 6 switched to use SSL by default and our web servers don’t yet take that fact into account. Searching still works fine, but in some situations the HTTP referer header isn’t passed on to the destination page. We’re investigating different options to address this issue.

As Google investigates different options to address this we’re left dealing with a serious data problem. Personally, I think Google Analytics should have a message within the interface that warns people of this issue until it’s fixed.

RKG did a nice job of tracking this and showing how to estimate the hidden search traffic. But for some reason this issue doesn’t seem to be getting as much traction as it should so I wanted to demonstrate the problem and show exactly how you can fight back. Because it’s tough enough being an SEO.

Organic Search Traffic Graph 2012

At a glance it looks like this has been a decent year for this client. But it’s actually better than it looks in October and November. Follow along to see just how much better.

Create iOS Advanced Segments

The first step is to create two Advanced Segments, one for iOS and one for iOS 6.

iOS Advanced Segment in Google Analytics

In May the labeling of Apple Operating Systems changed from specific devices to iOS. So include all four so you can see your iOS traffic for the entire year.

iOS 6 Advanced Segment in Google Analytics

The iOS 6 segment is straightforward and will only be used to demonstrate and prove the problem. Also, if you want to perform this analysis on multiple analytics properties be sure to save these segments to any profile.

The Scene Of The Crime

Once you have your advanced segments you want to apply them as you look at direct traffic by month.

Search Theft Underway

This plainly shows that direct traffic suddenly jumped from traditional levels upon the release of iOS 6 in late September.

Reclaiming Stolen Search Traffic

Every SEO should be reclaiming this stolen traffic to ensure they (and their clients) are seeing the real picture. Here’s my simple method of figuring out how much you should take back.

Three Month iOS Direct Search Ratio

I’ve taken a three month slice of iOS traffic composed of April, May and June. From there I’m looking to see direct traffic as a percentage of the sum of direct and organic. The reason I’m not doing direct as a percentage of the total is to reduce any noise from referral spikes, paid search campaigns or other channel specific fluctuations.

In this instance direct comprises 10.5%. If you want to go the extra mile and quell the OCD demons in your head (or is that just me) you can do this for every month to ensure you’ve got the right percentage. I did and am confident that the percentage for this site is 10.5%.

Be aware, it will be different for each site.

Next I look at November and perform the same calculation just to confirm that it’s out of whack. At 46.6% it’s clearly departed from the established baseline.

November Direct and Search Traffic for iOS

I simply apply the proper direct traffic percentage (10.5% in this case) to the sum of direct and organic traffic. That’s the real amount of direct traffic. I then subtract that from the reported direct traffic to find the lost search traffic number.

The equation is none-((organic+none)*percentage). In this case I just reclaimed 79,080 search visits!

Better SEO Results

Get the credit you deserve and apply those stolen search visits to organic traffic.

November Search Lift from iOS Search

A very quick calculation shows that reclaiming iOS search traffic produced a 4.6% bump in organic traffic for this client. That’s the best 32 minutes I’ve spent in a long time. Now it’s your turn.

TL;DR

Changes in how Safari searches are passed to Google Analytics is causing organic searches to be listed under direct traffic. Give clients the real picture and get the credit you deserve by properly attributing iOS traffic.

Keyword Match Ratio

October 27 2012 // Analytics + SEO // 35 Comments

That awkward moment when you realize you’ve been staring at interesting data for years without knowing it.

That Awkward Moment When ...

Every day you’re probably using Google Keyword Tool query volume in your SEO research. Of course you have to be careful to use the correct match type, right? You don’t want to make the mistake of promising broad match level volume to a client.

Recently I began to wonder about the differences in match type volume. Because they are substantial.

Keyword Match Ratio

What am I talking about? The keyword match ratio is the broad match volume of a keyword divided by the exact match volume of a keyword.

Keyword Match Ratio Examples

I know these are completely different keywords but the difference is pretty astounding. This metric should be meaningful. It’s not some end-all-to-be-all metric, but I believe the keyword match ratio is useful.

Here’s how I’ve been looking at and using the keyword match ratio.

Determining Intent

One of the main ways I’ve been using this new metric is in determining intent. Or, more specifically, is the intent uniform or fractured?

A low keyword match ratio indicates a more uniform syntax which often maps to uniform intent. In other words, there aren’t as many keyword variations of that term or topic. Uniform intent is great from a search perspective because you can more easily deliver a relevant and valuable experience for that traffic.

A high keyword match ratio indicates a less uniform syntax which may indicate fractured intent. That means there might be a lot of ways to talk about that topic or could point to a whole modifier class. Fractured intent is more difficult to satisfy since users may come with different expectations of value.

Unfortunately, determining intent got more difficult when Google reduced the level of category detail during the merge of Google Trends and Google Insights for Search.

Google Trends Category Data Limitation

You can still see that there’s potential fractured intent here but the old version would have presented the various percentage breakdowns for each category which was quite useful. Keyword match ratio provides a new way to validate whether you should be concerned about fractured intent.

Identifying Content Opportunities

The other way I’ve been using the keyword match ratio is to identify areas ripe for content creation. In this case, a high keyword match ratio indicates a potential for different modifiers and phrases for that keyword.

Hardwood Floors Keyword Match Ratio and Content Ideas

The term ‘hardwood floors’ has a pretty high keyword match ratio and even the suggested ad groups provide ample content ideas. Go a step further and use related searches and Google Autocomplete suggestions to get more ideas that match query syntax.

Hardwood Floors Related Searches

Hardwood Floors Google Autocomplete Suggestions

Look at all those content opportunities! Follow high keyword match ratios to uncover content ideas and opportunities.

Benchmarking

While I can usually just tell whether a keyword match ratio is high or low, or simply compare it to other keywords in a list, I wondered if I could create a benchmark. Enter Dr. Pete, who was kind enough to share the 1,000 keywords that comprise MozCast. (Thank you.)

The first thing I did was see how the keyword match ratio changed with query length.

keyword match ration by query word count

As you might expect, the ratio declines as the number of words in the query increase. I like when things make sense! What this allows me to do is identify specific keywords that are materially outside of the norm.

What about the 2 word query with a ratio of 226.3 or the 2 word query with a ratio of 2.2. The ratio tells you something about the behavior of that keyword. It’s your job to figure out what it is.

Competition

My next idea was to map the ratio to keyword difficulty. I experimented with using the competition number via the Google Keyword Tool as a proxy but the numbers were all over the place.

So … I generated the keyword difficulty for 92% of the list five painstaking keywords at a time via the SEOmoz Keyword Difficulty Tool. (There’s a 300 a day limit so I didn’t quite get through the entire list.)

Keyword Match Ratio by Keyword Difficulty Graph

There might be a trend there but it was difficult to tell with all the noise. So I rounded keyword difficulty into deciles.

Keyword Match Ratio by Keyword Difficulty Trable

No terms fit into the 0, 10 or 100 deciles so I removed those rows from the table. What’s left does seem to indicate a rising keyword match ratio with increased keyword difficulty. That’s interesting and makes a bit of sense too. Competitive terms often have more volume and likely have a greater number of variants.

Putting It All Together

The question is how you can use all of this information together? To be honest, I haven’t come up with the perfect formula but I find it interesting to take terms and see where they fall against these benchmarks.

Swedish Fish

What about the term ‘swedish fish’? This 2 word keyword has a keyword match ratio of 3.3, well below the 2 word benchmark. In addition, with a 41% keyword difficulty it falls into the 40 bucket, which again puts it below the standard keyword match ratio for that difficulty.

That tells me the intent behind the term ‘swedish fish’ is uniform and it might be an area where a well optimized piece of content could rank well. Yum!

A term with a low keyword match ratio and low competition is a great SEO opportunity.

The syntax and intent are clear and you can provide relevant and useful content to fill that need. Of course, all of this has to produce productive traffic. We’re not doing SEO just for gold stars and pats on the back, right?

Solar Panels

What about a term like ‘solar panels’? It has a keyword match ratio of 13.5, above the 2 word benchmark. With a keyword difficulty of 70% it also scores slightly over the average.

That tells me optimizing for ‘solar panels’ is going to be a hot mess. Instead, I’d want to look for phrases and modifiers that might be more attractive instead, with the long-term goal of building up to this head term.

Locate the specific intents and keywords that contribute to a high keyword match ratio and produce relevant content that satisfies and engages.

Context, Brains and Disclaimers

A couple of things you should know about the keyword match ratio. You need to use it in conjunction with other tools, in particular your brain. Context is important and different verticals and modifiers will have different keyword match ratio patterns.

So while I provide the benchmarks above you should be thinking about how the ratio fits into the keyword universe for your site, or for that particular modifier. If you were a coupon site you might want to see which store + coupons terms had the highest and lowest keyword match ratio.

There’s also the possibility that the set of data I used for the benchmark isn’t representative. However, I think Dr. Pete has done a pretty good job here and while some of the terms are strange and mundane that’s not a bad reflection of reality.

You’ll also note that I’m not doing any heavy duty statistical analysis here. While I understand and enjoy those endeavors I think pattern recognition can take you pretty far pretty quickly. Maybe someone else can pick up this thread and create something more statistically valid.

In the interim, I’m using the keyword match ratio as an SEO hack to help me find potential diamonds in the rough and areas for content creation.

TL;DR

The keyword match ratio measures the ratio of broad match volume and exact match volume. This metric is not fool proof. You need to use your brain when looking at it. But if you’ve got a good head on your shoulders the keyword match ratio can help you determine intent and sniff out content opportunities.

Google Analytics Y Axis Scale

June 20 2012 // Analytics // 6 Comments

One of the things that bothered me about the ‘new’ Google Analytics was the relative y axis. Google Analytics would chart traffic on a much smaller scale based on the time period and traffic volume.

Relative Y Axis

So if you had daily traffic between 12,000 and 14,000 visits the scale might be from 10,000 to 15,000. The result? Fluctuations in traffic appeared much bigger than they were in reality.

Top Secret Movie Big Phone Gag

This caused a number of people to panic. Frantic emails were sent. Even after they understood that the seemingly large drop in traffic was only 2% (and could be chalked up to a holiday weekend) the visual cue was unnerving. Information aesthetics matter!

Absolute Y Axis

I lived with (but didn’t like) the relative graphing feature. I mean, Google Analytics is a free product so I can’t get too worked up about it. But the other day as I refreshed one of my advanced segments the graph got all screwy and I had to reload Google Analytics entirely.

Google Analytics Graph with an Absolute Y Axis

The graph started from zero! Things looked ‘right’ again. Was this a permanent change? I reached out to Adam Singer who looped in Justin Cutroni who confirmed the return of the absolute axis.

We heard from a lot of people that the relative axis was sub-optimal. So the absolute axis is back!

I am very pleased that Google Analytics has reverted to the absolute axis and believe it conveys the information in a more ‘honest’ way. So, from one user, thank you.

2012 Internet, SEO and Technology Predictions

December 27 2011 // Analytics + SEO + Technology // 8 Comments

It’s time again to gaze into my crystal ball and make some predictions for 2012.

Crystal Ball Technology Predictions

2012 Predictions

For reference, here are my predictions for 2011, 2010 and 2009. I was a bit too safe last year so I’m making some bold predictions this time around.

Chrome Becomes Top Browser

Having already surpassed Firefox this year, Chrome will see accelerated adoption, surpassing Internet Explorer as the top desktop browser in the closing weeks of 2012.

DuckDuckGo Cracks Mainstream

Gabriel Weinberg puts new funding to work and capitalizes on the ‘search is about answers’ meme. DuckDuckGo leapfrogs over AOL and Ask in 2012, securing itself as the fourth largest search engine.

Google Implements AuthorRank

Google spent 2011 building an identity platform, launching and aggressively promoting authorship while building an internal influence metric. In 2012 they’ll put this all together and use AuthorRank (referred to in patents as Agent Rank) as a search signal. It will have a more profound impact on search than all Panda updates combined.

Image Search Gets Serious

Pinterest. Instagram. mlkshk. We Heart It. Flickr. Meme Generator. The Internet runs on images. Look for a new image search engine, as well as image search analytics. Hopefully this will cause Google to improve (which is a kind word) image search tracking within Google Analytics.

SEO Tool Funding

VCs have been sniffing around SEO tool providers for a number of years. In 2012 one of the major SEO tool providers (SEOmoz or Raven) will receive a serious round of funding. I actually think this is a terrible idea but … there it is.

Frictionless Check-Ins

For location based services to really take off and reach the mainstream they’ll need a near frictionless check-in process. Throughout 2012 you’ll see Facebook, Foursquare and Google one-up each other in providing better ways to check-in. These will start with prompts and evolve into check-out (see Google Wallet) integrations.

Google+ Plateaus

As much as I like Google+ I think it will plateau in mid-2012 and remain a solid second fiddle to Facebook. That’s not a knock of Google+ or the value it brings to both users and Google. There are simply too many choices and no compelling case for mass migration.

HTML5 (Finally) Becomes Important

After a few years of hype HTML5 becomes important, delivering rich experiences that users will come to expect. As both site adoption and browser compatibility rise, search engines will begin to use new HTML5 tags to better understand and analyze pages.

Schema.org Stalls

Structured mark-up will continue to be important but Schema.org adoption will stall. Instead, Google will continue to be an omnivore, happy to digest any type of structured mark-up, while other entities like Facebook will continue to promote their own proprietary mark-up.

Mobile Search Skyrockets

Only 40% of U.S. mobile users have smartphones. That’s going to change in a big way in 2012 as both Apple and Google fight to secure these mobile users. Mobile search will be the place for growth as desktop search growth falls to single digits.

Yahoo! Buys Tumblr

Doubling down on content Yahoo! will buy Tumblr, hoping to extend their contributor network and overlay a sophisticated, targeted display advertising network. In doing so, they’ll quickly shutter all porn related Tumblr blogs.

Google Acquires Topsy

Topsy, the last real-time search engine, is acquired by Google who quickly shuts down the Topsy API and applies the talent to their own initiatives on both desktop and mobile platforms.

Not Provided Keyword Not A Problem

November 21 2011 // Analytics + Rant + SEO // 15 Comments

Do I think Google’s policy around encrypting searches (except for paid clicks) for logged-in users is fair? No.

Fair Is Where You Get Cotton Candy

But whining about it seems unproductive, particularly since the impact of (not provided) isn’t catastrophic. That’s right, the sky is not falling. Here’s why.

(Not Provided) Keyword

By now I’m sure you’ve seen the Google Analytics line graph that shows the rise of (not provided) traffic.

Not Provided Keyword Google Analytics Graph

Sure enough, 17% of all organic Google traffic on this blog is now (not provided). That’s high in comparison to what I see among my client base but makes sense given the audience of this blog.

Like many others (not provided) is also my top keyword by a wide margin. I think seeing this scares people but it makes perfect sense. What other keyword is going to show up under every URL?

Instead of staring at that big aggregate number you have to look at the impact (not provided) is having on a URL by URL basis.

Landing Page by Keywords

To look at the impact of (not provided) for a specific URL you need to view your Google organic traffic by Landing Page. Then drill down on a specific URL and use Keyword as your secondary dimension. Here’s a sample landing page by keywords report for my bounce rate vs exit rate post.

Landing Page by Keyword Report with Not Provided

In this example, a full 39% of the traffic is (not provided). But a look at the remaining 61% makes it pretty clear what keywords bring traffic to this page. In fact, there are 68 total keywords in this time frame.

Keyword Clustering Example

Clustering these long-tail keywords can provide you with the added insight necessary to be confident in your optimization strategy.

(Not Provided) Keyword Distribution

The distribution of keywords outside of (not provided) gives us insight into the keyword composition of (not provided). In other words, the keywords we do see tell us about the keywords we don’t.

Do we really think that the keywords that make up (not provided) are going to be that different from the ones we do see? It’s highly improbable that a query like ‘moonraker steel teeth’ is driving traffic under (not provided) in my example above.

If you want to take things a step further you can apply the distribution of the clustered keywords against the pool of (not provided) traffic. First you reduce the denominator by subtracting the (not provided) traffic from the total. In this instance that’s 208 – 88 which is 120.

Even without any clustering you can take the first keyword (bounce rate vs. exit rate) and determine that it comprises 20% of the remaining traffic (24/120). You can then apply that 20% to the (not provided) traffic (88) and conclude that approximately 18 visits to (not provided) are comprised of that specific keyword.

Is this perfectly accurate? No. Is it good enough? Yes. Keyword clustering will further reduce the variance you might see by specific keyword.

Performance of (Not Provided) Keywords

The assumption I’m making here is that the keyword behavior of those logged-in to Google doesn’t differ dramatically from those who are not logged-in. I’m not saying there might not be some difference but I don’t see the difference being large enough to be material.

If you have an established URL with a history of getting a steady stream of traffic you can go back and compare the performance before and after (not provided) was introduced. I’ve done this a number of times (across client installations) and continue to find little to no difference when using the distribution method above.

Even without this analysis it comes down to whether you believe that query intent changes based on whether a person is logged-in or not? Given that many users probably don’t even know they’re logged-in, I’ll take no for 800 Alex.

What’s even more interesting is that this is information we didn’t have previously. If by chance all of your conversions only happen from those logged-in, how would you have made that determination prior to (not provided) being introduced? Yeah … you couldn’t.

While Google has made the keyword private they’ve actually broadcast usage information.

(Not Provided) Solutions

Keep Calm and SEO On

Don’t get me wrong. I’m not happy about the missing data, nor the double standard between paid and organic clicks. Google has a decent privacy model through their Ads Preferences Manager. They could adopt the same process here and allow users to opt-out instead of the blanket opt-in currently in place.

Barring that, I’d like to know how many keywords are included in the (not provided) traffic in a given time period. Even better would be a drill-down feature with traffic against a set of anonymized keywords.

Google Analytics Not Provided Keyword Drill Down

However, I’m not counting on these things coming to fruition so it’s my job to figure out how to do keyword research and optimization given the new normal. As I’ve shown, you can continue to use Google Analytics, particularly if you cluster keywords appropriately.

Of course you should be using other tools to determine user syntax, identify keyword modifiers and define query intent. When keyword performance is truly in doubt you can even resort to running a quick AdWords campaign. While this might irk you and elicit tin foil hat theories you should probably be doing a bit of this anyway.

TL;DR

Google’s (not provided) policy might not be fair but is far from the end of the world. Whining about (not provided) isn’t going to change anything. Figuring out how to overcome this obstacle is your job and how you’ll distance yourself from the competition.

Image Search in Google Analytics

July 26 2011 // Analytics + SEO // 19 Comments

Think you got a bump from Panda 2.3? Not so fast.

Image Search Analytics

In looking at a number of client sites I notice that image search traffic, tracked under referring traffic (google.com / referral) with the referral path of imgres, fell off a cliff as of July 23rd.

Where'd My Image Traffic Go?

Where did that image traffic go? Organic.

Organic Image Search Traffic Bump

So if you thought you’d been the beneficiary of Panda 2.3 (launched late last week), you might want to make sure it’s not a phantom image search bump.

The Definition of Organic

At present I can’t find an easy way within Google Analytics to distinguish between organic traffic that is search based versus image based. That strikes me as a step back since these forms of traffic are not homogeneous in nature. Lumping image search in with organic is like smearing vaseline on your windshield. I can still see, just not as well as I could before.

There’s probably a hack you can put together via filters, but most users won’t make that effort.

Where’s Image Search?

This isn’t the first time Google has played Where’s Waldo with image search. On May 6th, 2010 Google moved image search traffic from images.google.com to google.com.

images.google.com traffic drop

At least that time you could wander around Google Analytics and spot the new source/medium that would provide the same level of specificity. Oddly, you’d still see some stray images.google.com traffic after this change. I always meant to track that down but never got around to it. This new update seems to finish the job and eliminate the remaining images.google.com traffic that had been trickling in.

New Dimension Please

I am hoping that this is just evidence that Google Analytics will launch a new dimension so we can separate these two different types of search traffic. Yet, you’d think they’d launch the dimension before migrating the traffic.

For a long time I figured that these changes were an indication that image search was the ugly duckling of the bunch. But recent events make me believe that Google is very invested in image search, so why the lack viable reporting? No, ‘it’s free’ is not the right answer.

I’m waiting to hear from a few Google sources and will update this post if I get any type of insight or confirmation. Until then, how do you feel about this change?