top of page

Crushing It With Competitive Intelligence Analysis: Best Metrics, Reports


Half-way through this post, you'll seriously wonder why you've spent so much time obsessing with Adobe/Google Analytics/Chartbeat or other web analytics tool.

When you are done reading the post, you'll be super mad that your marketing strategy is not more influenced by your competitor's data!

Such is the power of being able to proactively identify which of your competitor's strategies are working well, where their current customers come from, and what specific tactics you should experiment with to create and advantage for yourself.

Regular readers of the blog know of my deep love for competitive intelligence analysis. My first blog post on the topic of CIA was on 14th Aug 2006! Competitive Intelligence Analysis: Why, What & How to Choose. CIA also formed one of five foundational elements in my best-selling book Web Analytics 2.0.

Since then, as luck would have it, we have more tools, they are smarter, and have richer data-sets.

In this post we'll go back to the wondrous world of competitive intelligence analysis. We'll look at an incredible cluster of examples, from the simple to the sublime, that will help you learn practical strategies you can immediately go back and apply in your role as a Marketer, Analyst, or the Boss of all Things Digital.

Here are the key elements we will explore in our quest to become CI Analysis Ninjas:


When I say this is the ultimate guide for learning how to do amazing competitive intelligence analysis, the above list should clearly demonstrate that I'm not kidding around. You are going to learn A Lot!

My recommendation is to make sure you read the foundational concepts/caveats first (and twice). Without it, you are not going to shed your imprecise assumptions/impressions/past beliefs, nor learn about new possibilities. After that first section, you are welcome to jump around if you would like to, though I do recommend reading one complete section, practicing it, and then moving to the next one. Bookmark the post, keep coming back to it.

Excited? I am so excited, come on!!

Foundational Concepts/Caveats.

It is incredible to realize that you can spy on your competitors. While not quite as sexy as actual spy craft, it is fun and insightful. But, there are some important things that you should be aware of/brush up on/deeply internalize. Or, you'll either abandon this fun journey too quickly or, worse, stay on it and suffer way more than you need to.

CI data collection.

How does any tool have access to your website or mobile app data? You did not allow them to tag your site, right? But in the lovely environment of the web there are a number of ways to collect your data. I strongly encourage you to read this blog post: Competitive Intelligence Data Sources & Best Practices

The post outlines eight sources vendors use to collect data: Toolbars, panels, ISP data, search engines, self reported, scraped/indexed, hybrid, and external voc. The value of competitive intelligence (CI) is huge, it is well worth investing in learning how the sausage is made. You'll suddenly understand why, for example, something like Alexa is a unwise investment of your time.

Always ask your vendor two questions: 1. How do you collect data? 2. What is the sample size?

Multiple sources, of the eight above, are better. The larger the sample the better.

Competitive intelligence data will never match your site's analytics tool.


Here's how I've often explained the two different data-sets. In one case, Charlie's standing at the door of your supermarket and every time a customer walks in, Charlie presses a button and records that entry. In the other case, Pooja is standing on the moon with a telescope and each time she detects that someone's walking into your supermarket, Pooja will press a button and record that entry.

Since Pooja's likely to miss some folks walking in (gosh darn clouds!), she will look at all her observations of other supermarkets on Earth, on other planets, and do her best to give your the best normalized, indexed, algorithm'ed number.

You, see why the two numbers might not tie?

If you do not possess inside you the capacity to see beyond this, it is important that you stop reading this post and head over to buzzfeed because that might be a lesser waste of time (though that is giving buzzfeed a lot of credit :)).

CI data is valuable because…You'll look at the same competitive space you live in. The numbers for you and your competitors will be roughly equally wrong. You won't look at specific differences (it says 3,256,293 for me and 3,256,193 for our competitor, we win!), you'll compare long-term trends. You won't obsess about the long-long tail (data that sparse is not confidence-worthy), you'll look for big fat head opportunities. And so on, and so forth. In all of those cases (outlined below), you'll derive amazing, actionable insights.

Perhaps thinking about CI analysis in this way will help… You are comparing slightly-bruised-apples to slightly-bruised-apples. And that is 100% ok. But if you can't get over Charlie not matching Pooja, I've mentioned the buzzfeed option above.

Competitive intelligence tools will never match each other!


A slight variation of the above.

Each tool uses their own data collection methodologies, their own data processing algorithms, their own assumptions, their own way to plug in any holes, and their own ways of "normalizing" the data-set to give you the best possible answer. They will never tie to each other (if they do, run, the zombie apocalypse is upon us!).

Here's a simple example.

This is Occam's Razor's data using a tool I really like and use a lot, SimilarWeb

similarweb kaushik

It is a profound over-estimation of reality.

Here is the data for this blog from another tool I really like and use a lot, Compete

compete kaushik

It is pretty close to reality, though everything from Dec '13 through mid-year is imprecise.

And here is another tool I'm sure you've heard of Hitwise

hitwise

It profoundly under-estimates my traffic.

This does not mean these tools are useless.

Hitwise, for example, has one of the largest datasets of this type in the world. It is the differences between how they collect and process the data that creates the differences.

My recommendation is to investigate many tools, and then stick to the one you are most comfortable with. You'll be surprised, especially by the end of this post, how much better your digital strategy can be from comparing slightly-bruised-apples to slightly-bruised-apples!

Small sites are out-of-luck.


There is one other thing going on in the three pictures above: All CI tools are pretty terrible at data they report for small sites (and heartbreakingly Occam's Razor is a small site).

It is a matter of how the data is collected. Their listening posts simply can't accommodate for the low signal in all the noise, no matter how many of the eight CI data collection strategies they use.

Here's a rough guide from my experience (as you do this more, you'll form your own filters).


If you have under 100,000 Unique Visitors each month, your data (or your competitor's) in any CI tool is mostly useless.

If you have between 250k-500k Unique Visitors, you are in a decent place.

Any site with more than 500k, to say 1 mil, is in a very good place.

Anything over 1.5 – 2 mil, you should start to see patterns in the CI tool you see in your own analytics data.

So, what do you do if you have less than 500k Unique Visitors each month? Do analysis on your big competitors! Learn from them (specific recommendations below). And, for now rejoice that your big competitor has nothing on you! :)

Small countries are out-of-luck.


Similar reasons. If you are in Croatia, you might not have very good CI data for your country (you'll have to make do with relaxing on your world-class beaches that are the envy of the world!).

Sadly, the same for perhaps the entire African continent. Central America. Others.

Sometimes there is no signal in all the noise and if you do hear a signal, spend time asking the two questions in the data collection section above.

So, what do you do if you are in a small country? Learn from what is happening in a like-type country larger then yours.

If you are too small, use industry sources to do ecosystem-centric analysis (more on in the last section of this post). If you are too small, maybe use surveys or primary research to learn competitive examples.

Site-centric CI vs. Ecosystem-centric CI.


As Edward Snowden and the NSA/CIA/MI6/GCHQ documents releases have shown, there are many kinds of competitive intelligence in the world. Our interest here is not that type of intelligence. [Note to NSA reviewer of this blog post: I'm a harmless digital marketer, totally not worth your attention.]

In our scenario there are two types of CI analysis.

Site-centric analysis is primarily focused on looking at direct competitors, or adjacent ones, and learning from what is happening with Acquisition, Behavior and Outcomes on their website (desktop and/or mobile).

Ecosystem-centric analysis is primarily focused on what is happening in the industry, in the sub-category, the small or big niche that everyone's playing in.

Both are valuable. In this post we'll focus mostly on site-centric competitive intelligence. I'll close with a fast sprint through ecosystem-centric analysis awesomeness, but it is not the main focus of this post.

Tools used in this post.


In this post I've used Compete, Hitwise, SimilarWeb for site-centric analysis. Google Trends, AdWords Keyword and Display Tools, Raven Tools, for ecosystem-centric analysis.

The common thread across all these tools is that I like them, regularly use them, and, if they are not free, have access to the Pro version.

The two exceptions are… I tend to dip in and out of my Raven subscription (primarily because I tend to dip in and out of SEO analysis needs). I tend to use Hitwise a lot less, or not at all, because it is a very hard to use, it is slow, the UX/UI, metrics and reports have not really evolved over the years. This is heartbreaking because the data-set, as mentioned above, is nice.

The other reason to use the above tools in this post is that you can get some stuff for free (Compete, SimilarWeb), or the full version at an affordable price, or you can get full version trial for a short-period (Raven), or you can get all the data completely free (Google Trends, Keyword and Display Tools).

There are many, may other tools in the world, wonderful tools. Please try them. My goal here is to only talk about what I know, use and like.

Important note: Compete is the only tool in this list that shows US Traffic only. It does not show data for non-US visitors to your site. And, if you type in an international site, like http://www.dhl.de, the data you are seeing is only for US visitors to dhl.de.

That is a lot of foundational elements! I'm sorry, but it is a complex topic and I want to make sure that when your boss questions you, or you are quizzed in a job interview, you are extremely well prepared to rock on this subject.

Enough talking, let's go do some analysis!

Traffic Trends Key Metrics Analysis.

The very first thing you are chomping at the bits to know is how much traffic your competitor gets! Easy question to answer, and free with Compete (remember, this is US visits only) or SimilarWeb (global traffic). So let's start there.

I've chosen to focus on one company for the entire blog post, for the sake of consistency and to make the lessons stick a bit more. We are going to look at Getty Images. I know some folks there, super-lovely people. Thought they would like to see an analysis of their digital strategy, and that of their direct competitors.

The first thing you'll do any CI tool is you'll likely look at your own data. I know you can't resist the temptation, go ahead, look at your data… here's Visits and Average Stay (Time on Site) for Getty in Compete…

compete getty traffic

Ohh… nice lines! You can look at the trend, see how it's going up and down. I also know that you can't resist comparing it your GA/Adobe data. I did recommend not doing that, but go ahead. I'll wait here. Happy now? Probably not. Are you going to listen to me now? Let's move on to doing productive things.

Remember, the best use of CI data is to contextually compare. Pretty much all tools have a button called Compare, and when you press that button you'll get something similar to this…

compete compare

I really like this feature in SimilarWeb and Compete. They'll both tell you who they think the competitors are (sometimes you see surprising one, and that is awesome!), and how closely they believe the businesses overlap.

All I do is add the top ones from the list, press Compare and…. boom!

compete getty competitive set

So much more fun. (You'll see my bias above, I prefer to use Unique Visitor comparisons rather than Visits.)

We (Getty Images) get very valuable context for our performance. We looked pretty good by ourselves in the first chart, now we understand how we are doing compared to a direct competitor-set.

Getty is the fourth green line. Fotosearch is tiny, and ignorable in this context. So we are dead last. Many times lower than Shutterstock. Good thing Getty also owns istockphoto (orange line)!

You can see that everyone's doing better recently. Right under the graph above, you can see some helpful raw numbers for recent performance. This can possibly make a good module in your tactical dashboard (we'll see cooler stuff we can add to our strategic dashboards later in this post).

compete traffic performance summary

Allow me to share an important tip. Every website in the world has a unique business strategy. Even direct competitors like Expedia and, say, Orbitz. It is important that you take this into account. Is Getty really competing with these sites, given its business model? Is it someone else? Or is this a gap in their business approach they should fix in some other way? These are good questions to ask when we undertake these comparisons.

So, what do you do with this data?


1. Compare long-term performance between competitors? Identify surprising peaks and dips, look for reasons why. (More on that below.)

2. Use this data to create benchmarks you can use for providing important context to your CEO when she/he looks at your own performance. Identify changes in your marketing mix/strategy.

3. Look for surprising competitors suggested by these tools. Why are you not worried about them?

Quite straightforward. Good way to dip your toes into the water.

Visitor/Audience Type Profile Analysis

Once you know how traffic trends are playing out, and you want to crush your competitors, you are surely curious about what kind of people are visiting your website vs. theirs.

No problem. You can look at the demographic, psychographic profiles.

Most competitive intelligence tools will have data that will allow you to compare Age, Income, Gender and other dimensions easily. This one's from Compete and I choose us and Dreamstime (as they are doing 2x better than us!)…

compete demographics

You can see that there are clear differences between the two sites. Remember my call to action on ensuring you understand your business strategy, this is where you can see it manifested the most.

Other tools, can provide more dimensions including children, political leanings, ethnicity etc…

demographics

You can see how this might be a lot more interesting for a magazine or content sites, and perhaps a lot less to other types of businesses.

On that note… If you search through the 300 posts I've written thus far (this one is 301!), and the 638,442 words in those posts, you'll notice that I've never, ever, never asked you to use demographic, psychographic data to do anything with digital. This is primarily because I fundamentally believe that this is almost useless data. On the web we can infer a person's intent using their actual behavior. Why do we need to know how old they are or if they are republican or female? If I can discern intent why do I need demo/psycho data? If I can infer that you want a Nexus 9 or are looking for an article on Libya or want to learn what a Roth IRA is, why do I care how old you are or if want to school? Using your intent, I can deliver the answer you are looking for! But, vice-a-versa is not true. If you filter by demographic/psychographic data, you will never engage a 75 yr old woman looking for an iPod (because you would already have ruled her out as old!). You can get inferred demographic data even in Google Analytics. It is substantially less useful than you might believe.

In the old world of TV/radio and no behavior data you had little choice. You had to rely on 300 surveys filled out by badgering people on street corners or when they are having dinners at home. You made do with what you had, and that was ok. But now, you don't have to do that anymore. It is 2015.

I'll get off my high-horse now. It is ok if you don't agree with me. I just want you to know why I believe what I believe.

What I find a lot more interesting (behavior!), is data like this one from SimilarWeb….

similarweb audience interests gettyimages2

It helps you understand the web wanderings of people who also visit your site (or a competitor's). What categories of content do they consume? What topics are covered in that category? What other websites do they visit?

I find this behavior to be a million times more useful than demographics or political leanings. And I'm totally ignoring that demo/psycho data is deeply inferred and algorithmized.

It is a lot less useful to know that 22 yr old, gay, African-American, republicans, who did not go to college, and have four children are 7% of the audience. It is a lot more useful to understand, this is what they search for, this is what they read, these are the sites they tend to visit.

So, what do you do with this data?


1. Your boss will absolutely love the demographics and psychographics the first time you show it to them. Do it. Earn some love. Why not? Then, slowly, try to wean them off this non-very-useful data.

2. Look for Audience Interests type data from your CI tool. Create hypotheses as to what this means for your content strategies on your own and rent digital platforms. Structure tests to validate these hypotheses.

3. Identify the impact on your display advertising campaigns. This will have direct implications on your ad targeting, ad content and ad landing pages strategies. Recommend changes.

Done right, you'll be surprised how effective #2 and #3 are. Both at improving your digital engagement and acquisition effectiveness, and at weaning your leadership away from the shiny hollow demo/psycho obsession.

Traffic Sources Competitive Analysis

You know how many people visit your digital existence, you have a better idea about who they are based on their personal attributes and their behavior.

Time to understand where they are coming from!

This is the biggest section of this blog post because it is often the area CI vendors will have most direct knowledge about people's behavior. Remember our metaphor above? Pooja is standing on the moon with a telescope and observing your supermarket, she will have some knowledge about what's going on inside the supermarket but she will know a lot more about where people were before they came to your supermarket, where they went after (she can see the trails more clearly), what search keywords brought them there, etc.

There is a lot of gold to mine here. Hence, a lot of your time will be spent in acquisition analysis to improve your digital advertising and marketing strategy.

Incoming and Outgoing Traffic

The highest level acquisition view you can get is knowing where people come to your competitor's site from, and where do they go after their visit. (For your own site, you don't have Outgoing data in your web analytics tool, so it is a nice to have from a CI tool.)

compete incoming outgoing traffic sources getty images

Look for competitors in the Incoming list (this also, won't be in Google Analytics or Adobe or WebTrends). Look at everyone and everything in the Outgoing list. If people are really happy with what they get on our Getty Images site, why do they go to istockphoto and Shutterstock next?

A great use of this data is to compare and contrast (with CI tools, because of all the caveats at the start of this post, it is good idea to always compare and contrast!). For example, what are the differences between Incoming sources for Dreamstime and Shutterstock?

compete incoming traffic sources dreamstime shutterstock

You can see clear differences between the two companies. They might have different business models, but this should be great external third-party data to take to your CMO and have her/him ask questions like: "Why is it that our second biggest source is Bing when our competitor's is Facebook?" There might be a good reason, but it is worth asking the question. Or for Shutterstock to wonder why Dreamstime seems to be so, so, much better at Google! (More on how to discover why that is the case, below.)

Naturally you want to dig below the surface. My favourite method is to use the Traffic Dashboard report in Compete (as I'd mentioned in my post on data visualization strategies and examples).

This is what it looks like, for our website and our images category leader…

compete traffic dashboard treemap

You can notice big differences in acquisition strategy right away – from the size of the boxes, and from, for example, seeing that Web Portals and Search is darker green (more growth) for Shutterstock than for us.

Beyond that, clever analysts that you all are, you'll notice subtle things like Shutterstock is likely better at building loyalty and getting people to come back again and again directly when compared to Getty. And many other such things are waiting to pounce on you!

If you are just starting out, you probably just want to know, what's their Miscellaneous and how can I steal more of that! Good question.

Just click on that box in the Treemap, and you'll get the answer…

compete traffic dashboard treemap-shutterstock-misc

Sweet, right? You can see the top sites, and you can see if in the last month traffic from that source grew or fell.

For me, the misc view is always a great place to look for relevant sites to build relationships with for acquisition (links, display ads, content, something else).

Another favourite of mine is the box below the Treemap in Compete where we can see cute things like these…

compete shutterstock top rising referrals

Saves me the effort of actually doing my job. :) It helps me find the top growing, declining, long-tail (#omg), social and search visits.

So, what do you do with this data?


1. Make big picture recommendations about changes to your company's owned, earned and paid marketing strategies. Stop wasting time with Social (or do more!). We need to re-evaluate our YouTube strategy. We need to shift our strategy to more dramatic focus on SEO. Etc. In each case, frame it as a hypothesis, test it, make bigger changes.

2. Identify tactical recommendations for your display, relationship-building, and other such strategies. Be specific, back it up with drill-down data from the Treemap view (or equivalent), and top movers (up or down).

3. This is much harder but try and figure out the implications of the Outgoing sources list. Are people going to that list of sites because they did not get what they want from us, are we more expensive, is our internal search broken, do we have the world's worst checkout experience (which is true of one of the sites we are talking about today)? This does require a much more strategic approach, master #1 and #2 before you go here.

Most advertising and marketing strategies are based on what we see in Google Analytics, or what we feel might work. Both have value. Additionally use CI data. Use what's already working for your competitors.

Search/Keyword Competitive Performance

Once you are done with the big traffic sources, it is time to focus on the source that is usually the single biggest: Search!

We will follow the same pattern, start macro, dig a bit, dig a bit more.

The very first thing you would want to track are the overall traffic trends. In this case, a simple distribution of overall search traffic and the amount that is coming via paid search. We are using Compete for below report, you can get a similar one in SimilarWeb (see what I did there? :)).

compete paid vs total

If you've learned one lesson thus far in this post, your site's data by itself is not all that interesting. You want context.

So here's us and the leader in our space…

compete getty shutterstock search comparison

Interesting! We were a bit closer last year, we have both grown since then, but our competitor has shown much better growth since then. Boo!

The next question is obvious, let's take a look at the traffic from paid search vs. organic…

compete getty shutterstock paid search comparison

Big difference here. You can see a specific shift in strategy starting May 2014 for Shutterstock, while we have continued to plod along at the same pace. Ouch.

I believe that it is not all that important if your competitor is beating you with paid search or organic search. If they are beating you with search, you have to take action! I've heard many a senior business leader trot out the excuse but they are beating us with paid, we don't have money to spend! My reply to them is: If search traffic is profitable for you, and PPC delivers profitable traffic, why would you let your competitor win? It is really that simple, and yet so hard for some executives.

If you compare the two graphs, Shutterstock is beating us in search overall by approx 1.5 million Unique Visitor. The delta in the above paid search graph, around 300k, is just a part of their winning strategy (which clearly is SEO + PPC, as it should be). We have our work cut out for us.

Then, defying every excuse from our executives for not doing the right thing, we shift to competitor crushing mode!

The first stop should not be the tsunami of keyword data every tool will provide you. In your CI tool, try to look for a feature that will allow you to filter and focus.

In Compete this is using the various filters they've already created. High traffic keywords. Keywords that bring a lot of people who also spend a lot of time on the site. Keywords with high organic share. So on and so forth…

compete search analysis options

It helps bring a lot of initial focus to your efforts. And trust me, you will need it.

Once you identify your criteria, you can dive deeper into the data. The columns might look a little different, but in general this is the type of information you'll see…

compete keyword details getty

It is easy to figure out what the key phrases are, why the competitor is winning, and how much more valuable one keyword is compared to others. I find both the paid and organic (or natural as Compete calls it), and the Average Time Index to be valuable in drawing up my initial world domination plans.

Similar report from HitWise. It does not have time, but you can see the other metrics are roughly similar…

hitwise search report gettyimages

I recommend taking the above mentioned keyword research approach first, learning from your couple biggest competitors, and doing this one second: Death match!

You type in your site, you type in your competitor and before you can say Tom Brady, you've got data!

compete keyword details getty-shutterstock volume time

I really like that it shows if you win on volume (amount of traffic you can get vs. your competitor) AND gives you a sense for quality by sharing how much time that traffic spends on the site.

So, you can see that there are some cases where we, Getty, are winning on volume, but for this bunch, we lose every time when it comes to getting people to stay on our site (even when we win!).

It is quite possible that Shutterstock is the suckiest website in the world and the time advantage exists for them because the site is so much more painful to use. It is easy to check, type the keyword into Bing, go to their site, see the experience. Now that you know how good it is, feel shame for coming up with the excuse for losing the time advantage, and resolve to win by making V for Vendetta (!) your new mantra. :)

Now you know both keywords to include into your SEO and PPC awesomization strategy, and the landing pages and experience to fix. How lovely.

In my quest to make our lives easier, I wanted to share a couple other features I adore.

In SimilarWeb, I love this little button you'll see on top of the search terms report: Exclude Brand Keywords.

similarweb filtering brand keywords

It is so useful! The biggest opportunity often is not in the brand keywords, it is in the category and long-tail keywords. And these reports are so often clustered at the top with 152 variations of the brand names. I hate it so much because I have to manually filter to find something useful.

Now, I just click on that button and magically, with no more clicks, most of the brand terms disappear. You are left with the category keywords I'm so interested in, and some mis-spellings of the brand terms which are much easier to deal with.

A second feature that is truly awesome, is in the Competitors folder. It's the Keyword Competitors report.

The first tab shows the businesses you compete with in organic ranking, people taking away your God given right to capture 100% of the clicks for all your brand, category and long-tail terms. :) Here's the report for Getty Images…

similarweb search organic competitors

It is so nice to have this data so easily accessible.

You'll see some obvious things, but you'll also notice clever things. Shutterstock is #7 in organic search competitors list for Getty Images. But Getty is #10 for Shutterstock! The top ten competitors for Shutterstock only have three in common with Getty. Hmm… weird. These types of analyses help you understand who your audience is, and that of your competitor. It also helps you figure out how to make very specific competitive choices about who (/what cluster) you will focus your SEO strategy on.

We often make those choices blind (or worse, based on what the HiPPO believes). At best it leads to less winning, at worse it causes your business to suffer big time.

The second tab of this report shows you the same data for paid search…

similarweb search paid competitors

The first one is Gettyimagesbank.com, the Korea based part of Getty. This should be an easy check, if they are really competing with each other, or apply segmentation to find geographic root-causes.

I'm sure you'll notice that the list in paid competitors looks different from the organic competitors. This is extremely common. Yet, most companies rarely have a search strategy that accommodates for this reality. Now, you don't have to be one of those companies.

You can also identify valuable insights like you compete a lot more with some companies in paid search (for example, Fotolia and Offset), and you compete with different ones in organic search. Now, you can have a smarter strategy. You can click on each of the sites and drill down into their detailed data, and from there to their own deep search analysis. And discover things like Offset is owned by Shutterstock! #arrrrhhh!

So, what do you do with this data?


1. Create benchmarks for Overall and Paid search traffic patterns of your competitors to create new and improved goals – month-by-month – for your own macro search strategy. Use these to make specific recommendations for search budget (to include people, process changes and ad spend).

2. Change your head and long-tail SEO strategy (links, content, distribution) based on your analysis of not just Share of Search or Volume metrics for keywords, but using engagement/behavior metrics to ensure your target list is pure and refined gold. Deeply analyze your organic competitors to get smarter insights (rather than random we think we like these keywords or you should rank for those other ones based on content you have – forget content you have, focus on what's working for the competition, let that have a say!).

3. Identify a very precise list of paid search keywords using both your paid competitors, and their most successful keywords. Ensure you include opportunities identified by your current head-to-head death match with your top competitors. Remember, winning paid search should include not just keyword lists but also a reflection on what ad targeting is working, the ad content and landing page strategies.

Search continues to be one of my absolute favourite ways to win short and long-term. (Email marketing is another fav.) There is perhaps no richer competitive intelligence dataset than for Search. Use it. Especially important in a world of Not Provided where your web analytics tools are empty.

Display Advertising Opportunity Analysis

Most competitive intelligence tools don't have anything for display advertising. This is sad, primarily because if you create great display ads you have a near amazing ability to engage potential customers at every stage of the See-Think-Do-Care universe. Not only that, with all the power that is coming to fore with programmatic buying, you can get so very close to getting the right message to the right person at the right time.

Only people on their way off the cliff now execute digital display strategies of the past which can be summarized as: Spray and pray!

With SimilarWeb you can get a sense for how your competitor is doing with display advertising. You can get the overall traffic numbers…

similarweb display advertising trends

Which gives you a sense for how strategically you are treating this channel, and how your competitors are doing it.

In isolation our graph looks like are on a rocketship! But in comparison to Shutterstock, even after they seemed to have pulled their pedal off the accelerator, you can see we are at least 40 times smaller!

From there you can dig a bit deeper to understand what platforms your competitor is using, and what categories they are targeting their display ads…

similarweb display advertising shutterstock details

And from there you can drill down into each category to understand what domains they focused on.

similarweb display advertising shutterstock sites

As usual it is less that you have to completely copy their strategy, and more that this gives you a good idea of what they are doing, and often what is working for them. You use that as context to create your own strategy.

So, what do you do with this data?


1. Quantify the implications of the gaps in your current display strategy, or lack thereof. The first graph should do fine.

2. Create a specific and targeted list of categories and domains you should target influenced by where your competitors are (and where they might not be). Especially analyze the companies beating you at Paid Search. Also leverage the list from the Compete Treemap view of incoming sources. For each category, you need to partner with your creative teams on ad content, with your marketing team for ad targeting. That is how you avoid spray and pray.

3. If you want to get your company to use display advertising in a smarter manner, and not fall for old cheap tricks like View-Thrus, it is critical that you understand Multi-Channel Attribution Modeling and ensure that you are using both Assisted Conversions and Last-Click Conversions to measure success. Oh, and if you are using both Macro and Micro-Outcomes as well, be careful for I just might run you over with a big hug and a thousand kisses!

Pretty straight forward. Even if it is a lot of work, across multiple teams.

Social Strategy Effectiveness

We can't forget Social can we? That's what the hype is all about. Even if organic social exposure on top channels are approaching sub 5% of your followers!

I'll recommend this report from SimilarWeb to be the cornerstone of your strategy…

similarweb social getty shutterstock

It highlights a head-to-head comparison of how you are doing vs. your competitor at a macro level and on specific social channels. The metric being displayed is Visits.

While I like the top right (you can get it from the traffic sources section above as well), I really adore the bottom part of the report. It shows Traffic Share and distribution of the share.

If you click on one of those referring social pages, you get the actual pages!

similarweb social shutterstock details

This is quite helpful because now you can click on these links and understand what they are doing on YouTube specifically that is driving traffic to their site. Or on reddit or vk.com. Awesome sauce.

So, what do you do with this data?


1. Identify channels that are more effective for your competitors. Contrast this with your strengths. Make re-allocation of people/money recommendations to your leadership team.

2. Content analysis via use of the second report above is perhaps your bff in trying to change your leadership's mind to shift from shouting to conversation marketing on social channels. Deliver five specific social efforts that your competitors have rocked, that you could possibly emulate (or do one better).

3. If your data looks like above, you'll quickly realize that you are among the 99% that stink at using one of the best social channels on the planet for businesses: YouTube. Leverage my Reach, Build, Engage framework for YouTube and identify current weakness in your strategy and recommend fixes.

Social media continues to be ROI positive on the long run if you are willing to invest in conversational marketing. If you are looking for short-term revenue or quick wins, you are better off skipping it (do something boring, but profitable, like Email or Affiliate marketing).

Geographic World Domination Opportunities

We've looked at so many dimensions related to where people come from. One last one remains, geography.

Most tools have this data available, here's how it looks like in SimilarWeb. I've taken the liberty of pasting two reports together…

similarweb geographic key metrics

There is not a lot more interpretation required. Where are you winning, where is your competitor winning.

Between direct competitors the level of stark difference you see above is rare. There seems to be little on common between Getty and Shutterstock. Even more surprisingly, the US traffic seems to be so low for Shutterstock. For Getty, this should lead to a big review of the current marketing strategy. Especially since Shutterstock is 4x the traffic size as Getty.

The world we live in is interconnected. But, more than that, the cost of providing digital only services (like photos, or literally anything in the cloud) is extremely low. There are few remaining decent reasons for any company to be country of origin centric.

So, what do you do with this data?


1. Provide a list of target countries to your management team where new opportunities exist (as identified by your competitors). At the minimum, make recommendations for decent sized experiments to validate profitability.

2. Do a little competitive intel of your own. For just a few dollars a month get VPN software. Switch your IP to the first opportunity from #1. Visit your competitor's site. Look at popular categories. What is selling more/less (usually available in a straightforward manner). Identify specific categories your company can enter into initially. (You can do this for any type of website, not just photos.)

3. If your CI tool provides City/DMA level information, use that to be more precise about where to target your advertising in these new opportunity geographies.

This analysis is not very complicated. Usually requires some grunt work.

Notice in the screenshot above, you not only get traffic data, you also get other delightful behavioral metrics. Good time for leaving acquisition behind, and moving on to onsite behavior.

Onsite Behavior Key Metrics Analysis

No matter how powerful the CI tool, observing deep behavior inside a website is hard. Unless the site has tons and tons of traffic, the these metrics can be a guess. See my guidance about traffic size and confidence at the start of this post.

Usually you will get four or five behavioral, what happens after a person lands on the site, metrics in a CI tool. Bounce Rate, Time on Site, Pageviews per Visit and Visits per Person. Here's a composite of Compete and SimilarWeb so that you can see how it will typically look…

compete similarweb site experience metrics

Visits per Person is particularly difficult to get. It is nice to have that as a measure of loyalty to the site.

(I'm pasting two different tools above to remind you, again, that this data rarely ties. You need to stick to one and then…)

Always make sure you contrast the key metrics with that of a competitor! Above data is for Getty, and these numbers for for Shutterstock…

engagement shutterstock

You can not the raw differences, and also, on the right, how these trends are performing over time (month/year/longer). That is where the key insights are.

Here's a screenshot of behavior metrics compared across multiple competitors in SimilarWeb…

similarweb pro site behaviour performance

What is up with Fotosearch's bounce rate! Yes, yes, their bigger problem is traffic, but still.

So, what do you do with this data?


1. Look for big deltas – positive or otherwise – in behavior performance. The numbers won't have too many insights for you, hence visit your competitor's websites and identify reasons for why Visit Duration or Pages/Visit might be long or short. Use their top search/display/social campaigns to visit their landing pages to identify why bounce rate might be high or low. Use these to make recommendations to your user experience team.

2. Time on Site and Pages/Visit are a bit dicey, you never know if a high number is good or bad. But, you can use Bounce Rate and Visits per Person to create benchmarks to index your own performance.

3. It is particularly useful to look at behavior metrics a couple months after your key competitors have made big changes to the UI or UX of their site. They probably won't tell you how it's going, but you can come check it out in your CI tool. :) Your insights will be of value to your user experience team.

I'm sure you are wondering why you don't see something like Conversion Rate in these reports. The reason is quite simple: It is hard/impossible.

As sessions switch to https (secure experience), it is difficult for prying eyes to peek through and know that a conversion happen. Honestly, it is hard to even know if someone added to cart. It is not totally impossible. One of the methods to capture CI data is to use intrusive monitoring software (Nielsen, ComScore, others use this strategy). Under this method with some work you can see a conversion. Just be careful about it. Use your good taste and common sense to identify the impact of sample size and sample type bias in such data.

Content Consumption Competitive Analysis

I'm sure the question on your mind is, what are all these Pages per Visit? What is the Time on Site being spent on?

This is where we hit into some limits. It is hard to get page level reports, for the reasons I've mentioned in the above section and at the top of this post.

But there is one report you can perhaps get some value from. The Sub-domains report.

Not every website is organized optimally for us to benefit from this report, but if it is, then you'll get something like this for Dell.com.

compete pro dell sub-domains

It gives you some sense for what content is being consumed, and the amount of change. It might take you visiting your competitor's website to get more context behind these rows of data.

SimilarWeb also provides this analysis. They provide % Traffic Share of each sub-domain – roughly equivalent to Unique Visitors above.

You can also look at the top sub-directories, and drill-down into the top pages in that sub-directory…

similarweb popular pages report dell

The data here, in pretty much all tools, is a bit iffy. But it does give you some sense for popular pages (insperion laptops are more popular than xps laptops).

So, what do you do with this data?


1. Most of the time you'll see that this data is not very useful. But if your competitor has been sweet and organized their sub-directories optimally, it is a worthy investment of time to infer their strategies when contrasted with yours. You might recommend prioritizing certain content higher to your leadership team (both for creation and advertising).

2. If you see anything decent at a page-level, that could also help provide specific insights to your website team.

I do wish we had a lot more here. But, we don't and while you might not agree with me, it is kind of nice that some data our competitors can't get to. : )

#OMG Mobile, Where's Mobile Data!

This past thanksgiving 52% of all online traffic was mobile per IBM. Mobile phones are being used early in the consideration process, they are driving influence. Tablets are driving purchases. Christmas was even better for Mobile (traffic, sales, store visits, everything). The next billion people who come online will only have mobile devices.

I believe that this year mobile will account for a majority of the site traffic (greater than 51%) for commercial websites, and stay majority forever.

I share that as context for the fact that many competitive intelligence tools don't have any mobile data as of now. Or in the rare cases that they do, it is such a small sample size that it is completely untrustworthy. So be aware that this casts a pall of gloom over our CI data.

[This gloom, mercifully, does not extend to the Google ecosystem-centric competitive intelligence tools below. You can segment that data by desktop or mobile. #hurray]

As you use CI tools, and perform the wonderful analysis above, check your site's digital analytics tool for your mobile-desktop split. That should serve as guidance regarding how much might be missing from your CI tool.

Ask every single CI vendor you use: What is the sample size of your mobile data, what is it for desktop? Encourage an honest answer. If you have the power, put immense pressure on them to fix this issue on a priority basis. I'm doing this every other Tuesday.

SimilarWeb does clearly label their data, big props for them for this transparency…

similarweb desktop audience overview

And, this is very better, for each website they have a completely different report for mobile traffic. It provides us with some key insights.

First, you can see the long term trend in mobile and desktop traffic….

similarweb pro mobile traffic gettyimages

Second, they also provide mobile and desktop data split out for the four key metrics that they have in rest of the tool….

similarweb pro mobile behavior gettyimages

This is extremely helpful as you can see huge differences in behavior. You might have expected this difference, but it is nice to see it quantified.

Finally, what is a good CI tool without the ability to compare and contrast. You can create a report to analyze the trend in mobile traffic (Average Monthly Visits) for yourself and your competitors…

similarweb mobile competitor analysis

You can see that the difference between Shutterstock and all other competitors is the starkest for mobile. Shutterstock is doing so much better. Good food for thought for everyone else. Or a swift kick in the… you get it.

At the moment this is all we have in SimilarWeb (let me note, it is much more than anyone else for now!). As they evolve, hopefully we can see other valuable acquisition metrics and dimensions to enable us to do smarter mobile analytics.

Ecosystem-Centric Competitive Intelligence Analysis

A quick refresh from the very top of this post: Site-centric CI is primarily focused on looking at direct competitors or adjacent ones. Everything we have done so far (above). Ecosystem-centric CI is primarily focused on what is happening in the industry, sub-category, that everyone's playing in.

I'd promised we would take a quick whirl through it (mostly because I would not be happy if you did not know this). Brace yourself for a fast ride!

Ecosystem analysis will start, usually, by looking at the category we are playing in. It is likely you are tired of the Photos space, let's look at something more fun, Jewelry!

You'll initially want to know how the category is doing. Go to Profiles, then Category, pick yours. (One of the 252 Industry Categories is… "Competitive Web Analytics"! It lists 125 websites, but not this one! Boo! Compete! Boo! :))

This is what the report will look like…

compete ecosystem analysis

You know the broad trends in your site traffic, now you can see how the industry overall does. (Is it surprising that there is such huge peak in Dec but barely anything on Valentine's Day?)

You can also note of the "available traffic" metrics, Unique Visitors and Visits. And are your numbers showing the growth you see there? They should, right?

You can scan the consumption metrics (pageviews and total time spent – 246 million!), but then ignore them as they are quite useless aggregated at this level.

Next, we drill down to understand what are the top x sites in the industry….

compete ecosystem analysis details

You have a much broader view of the world, because your starting point was different. Ecosystem-centric CI opens up more Unknown Unknowns.

Another one of my favourite ecosystem CI tools is Google Trends. If you are going to analyze organic search behavior, why not directly go to the source and exhaust it's value before you pay for a differen tool?

I love analyzing broad trends in intent using Google Trends. Here's one that I might be interested if I was running a food company, and wanted to know what to bet the farm on… Comparing user interest in tofu, seitan, tempeh and hummus…

google trends vegetarian food trends

Look at hummus go! It is the next salsa. :) On a serious note, this data is really good for looking at long term trends and identifying seasonal interest patterns. Try running a query with your industry name, and the top brand names in your space. That is how you get Share of Search, and amazing insights.

If I manufacture both tofu and hummus, I can use the Geo report to precisely understand where the indexed demand is coming from and adapt my marketing and advertising strategies accordingly. Currently my tofu ads are in heavy rotation in Pennsylvania, Colorado and Michigan. Oops…

google trends tofu geo

Compare and contrast with hummus… A significantly more diverse consumption pattern…

google trends hummus geo

Looks like everyone in America loves hummus. (I'm so surprised that California is missing from the top 7. That is the value of querying a database of intentions like Google Trends, you realize how wrong your gut is.)

Close out by understand the fastest rising queries for both….

google trends fastest rising searches

Use this data for your SEO, PPC campaigns, to invest in creating a new flavor of hummus (can anyone confirm if cauliflower hummus is any good, it sounds a bit yucky), and even, as you can see above, celebrities you should associate with.

If you want to learn more about other amazing analysis you can do, checkout my post on Google Trends (under it's old name): Google Insights for Search: Competitive Analysis.

Raven Tools is another great ecosystem analysis tool, though they have evolved to be a lot more (including your single repository for all your Google Analytics, Social and other data). I've used their Research Central to do website and keyword research. Here's a sample report for a brand I utterly love and adore… Patagonia…

raven tools research central patagonia

One of the really cool thing about Raven Tools is that it pulls data from many sources, Google, Majestic, Moz, Calais and others. This allows you to dig deep into data quite easily.

If you are purely interested in Paid Search effectiveness, the AdWords Keyword Planner is an amazing resource. Lots and lots of search behavior data laid bare for your consumption…

google keyword tool tofu keyword suggestions

While it was built for PPC, you can easily see how you can extract numerous insights for a more effective SEO strategy. Primarily because your Google Analytics/Adobe/IBM data only shows you how much you got, here you can see how much traffic is available!

You can also drill down…

google keyword tool tofu keyword trends

And glean a strong understanding of seasonality which is useful for every type of paid and organic campaign you want to run – including display and social.

Speaking of display… When it comes to finding where you can find audiences who might be most interested in your products or services, you can take advantage of the free Google Display Planner.

Continuing the fun we are having in ecosystem analysis, I give you Kim Kardashian!

google display planner

You would do this for your own site, but what we are looking at is the analysis of where to run advertising if we were in-charge of www.people.com and our customers were interested in Kim Kardashian (#doh).

You can of course use these lovely recommendations to figure out where to optimally target your display ads (look at that mobile number!), but also to more broadly understand attributes of your audiences, the sites they tend to be at and so much more.

A final recommendation, one you might not think fits in this category.

Google Analytics has benchmarking reports for your key metrics. The GA team pulls together analysis from your industry vertical, country and like-sized websites. These end up as benchmarks from the ecosystem that are integrated into GA and available for you to extract insights from.

Here's how it looks like in your channel grouping report…

google analytics benchmarking detailed report[1]

It is important to take advantage of this ecosystem CI data so that you can have stronger context to your performance. To learn, extensively, how to take advantage of this data, see the post.

I hope this gives you a feel for the breadth and depth of ecosystem-centric competitive intelligence analysis you can perform. With just a little bit of work, you can glean spectacular insights.

That's all of it. Everything I know, and love, about competitive intelligence analysis.

It is my fervent hope that you will fall in love with it too, if you don't already use this data. And if you do, I hope you learned some new tricks and new tools.

As always, it is your turn now.

Are you convinced that CI data can provide incredible context and insights to your site analytics data? Are you just a little bit less in love with Adobe/WebTrends/Google Analytics? For site-centric analysis, do you have an absolute favourite go-to report for acquisition or behavior? What insights have you found to be actionable in traffic reports? Is there valuable context you need for Search from CI tools that we've not covered here? We rushed through ecosystem-centric analysis, but is there one analysis you are now newly determined to perform?

Please share your advice, favorites, critique, and life lessons via comments below.

Thank you.

0 views0 comments

Recent Posts

See All

Bình luận


bottom of page