Improve your Web Analytics Reports with Compelling Comments

Do you ever feel like you spend lots of time working on web analytics reports that nobody reads? The problem could be too much data and not enough text. At one client, I recently discovered the power of comments to make my weekly web analytics reporting more engaging.

The Blank Stares

We’ve all had those meetings where you just can’t seem to connect with your audience. You spent all day preparing the web analytics report. It is packed with actionable insights that the company can use to improve click through, conversions, return on ad spending, etc. However, when you walk people through it, their eyes glaze over and they start checking their phones. When you ask if anyone has questions, all you get are blank stares.

You might have been presenting exactly the data that they asked you for. It might even answer exactly the questions that were being passionately debated in that marketing meeting last week. So, why is nobody paying attention? Where did all that intense interest go?

Most People Do Not Engage with Data

Beyond scientists, accountants, and web analysts, most people don’t get highly engaged when looking at data. At least that has been my experience. Even fancy charts don’t help that much.

What does help is text. Good comments, titles, and metric names. This text forms a narrative for the data that the audience can engage with. The narrative creates context that gives the data meaning.

As web analysts, we need less context than most people. We look at the data and infer the context and meaning. But, don’t assume that your audience thinks like you do. Some of them might, but a majority probably will not. They will not infer the context by looking at data. And without context, your data will not engage them. They will simply not receive the information being presented.

This is true even if your chart or table is showing your audience something critically important that they care deeply about.

Real-World Example

Every week, I prepare a report for one of my clients. This report contains a standard set of charts and tables that I include each week. It also contains some special ones that deal with issues of the day. Before I discovered the importance of narrative, I struggled with the “issue of the day” charts. Those charts produced a lot of blank stares.

A couple weeks ago, the “issue of the day” was concern over the amount of video plays happening on the website. The numbers seemed too low. Visitors were coming to the video section of the site, but not playing the videos. There were a bunch of theories about what the problem might be; and some very heated discussion with strong opinions.

As I was preparing my report that week, I investigated the video plays to see if there were some segments that accounted for more or less plays than others. I discovered some very interesting behavior, and here is the chart that I put together to summarize what I learned.

If you are a web analyst looking at that chart, you almost immediately think to yourself, “Wow, something is seriously wrong with video plays on mobile devices!” You probably notice the following right away:

• There is a huge difference between the play percentage on desktops (Windows, Mac) and mobile devices (Android, IOS).
• The play percentage is trending up on desktops, but down on mobile devices.

Immediately after that, you probably start speculating about what might be causing this. Several hypotheses might come to mind:

• Their video player is buggy on mobile platforms.
• The user interface for playing video is confusing on mobile platforms.
• Google analytics events aren’t being fired reliably on mobile platforms.

I placed this chart in my weekly report, at the top of the “New Insights” section. Since I was in a bit of a rush, I didn’t write any comments for it. It seemed pretty self-explanatory to me anyway.

When the weekly meeting started, we reviewed the standard sections in the report. There was some lively discussion, and time started running out. Eventually, we got to the “New Insights” section, and the client team looked at the Play Percentage chart above. I explained that this showed a big discrepancy in the likelihood of playing a video between a mobile visitor and a desktop visitor. I asked if they were aware of any problems with how the video player worked on mobile.

Silence. No reaction. Nobody had any comments or questions. At that point, the meeting leader had another appointment, so he ended the meeting and thanked me.

“What just happened?” I thought. Last week, this same group was highly engaged in discussing the low number of video plays. Increasing video plays seemed like the highest priority. Here, I just showed them a chart that clearly shows that the problem is on mobile devices, and nobody cares.

What Went Wrong

If you find yourself in a situation like this, where you have presented vitally important information, but nobody seems to care, think about how you might improve the report with comments.

Your audience probably does care about the point you are trying to make, but they didn’t engage with the data you presented. Because they didn’t engage, they did not connect your information with a business issue that they care about. Since they are all busy with other pressing matters, they quickly moved on to thinking about something else.

You can solve this problem by providing comments to narrate your data and relate it to a business problem that your audience cares about. And not only comments are important. Also, think carefully about the title and the names of the metrics you are using. All this text provides the context that is needed for your audience to engage with the data.

Define All Terms – Nothing is Obvious

Sometimes, you need to use metrics that your audience isn’t familiar with by name. Most often these are ratios showing the percentage of some important behavior relative to something else. A familiar example of a ratio is Average Pages per Visit. Your audience might know that one, and a handful of others. But the small set of metrics they know will not be enough for you to communicate effectively. Some of the time, you will need to introduce an unfamiliar metric. When that happens, make sure you include text that explains the meaning and importance of the metric.

In my case, I was using a metric that I called the “Video Play Percentage”. This is the ratio of plays to video page visits. It measures the likelihood that a visitor viewing a video page will actually play the video on that page. (Defined in Footnote 1).

On the client’s website we were capturing video plays using Google Analytics Event Tracking. Unique Pageviews are available in Google Analytics out of the box. In a custom report, I used an advanced filter to restrict Unique Pageviews to sum up only over the pages that contain video.

The analysis was solid. However, my audience did not know what “Video Play Percentage” meant. During the meeting, I gave them a quick explanation, but I don’t think that it really sunk in. Although the concept of a “Video Play Percentage” seemed obvious to me, it wasn’t obvious to my client. Because they did not immediately grasp the meaning of this metric, they did not engage with the chart that I had prepared.

I decided to re-create the chart and provide comments to define “Video Play Percentage” and describe its importance. You can jump ahead and see the reworked chart below.

You may notice that I actually put the mathematical definition of the ratio in a footnote. The definition that you need to put in your comments is a description of what the metric measures and why it is important – not how it is calculated. As a web analyst, you may think that the mathematical definition is critical to understanding the meaning of the metric. However, most of your audience probably doesn’t think that way. They will want to know “what is the point of this metric”. “The point” is provided by a simple description of what it measures and how it relates to a business issue.

Explain the Meaning of the Segments

Just like you need to explain an unfamiliar metric, you should also explain the segments used in your analysis. Do not assume that the meaning of the segments is obvious. Your audience may be familiar with some basic segments (like Male and Female), but don’t assume more than that.

In my case, I had segmented the data by operating system. Looking back, I realized that the relevance of the operating system might not have been readily apparent. In truth, while I was digging into the data, I was just trying out different segments, to see if poor performance jumped out in any one place. When I looked at the data by operating system, I realized the big difference between the mobile and desktop operating systems.

What I didn’t realize was that my audience might not look at the chart and immediately see that iOS and Android were underperforming, and therefore the video play problem was on mobile devices. Again, it might be obvious to you, but you need to connect the dots for your audience.

To solve a problem like this, you can explain what the segments are, and what they mean. Or, you can combine the segments into bigger groups, name the groups something familiar, and simplify the chart.

That’s the approach that I took. I combined the operating systems, so that I just had two segments – desktop and mobile.

Relate Your Insight Directly to the Business Problem

Most importantly, you need to provide context to help your audience relate the data you are showing them with a business problem that they are already concerned about and familiar with.

In my case, the problem that the client was familiar with was that there were “not enough video plays”. My data showed that the cause of this problem was poor results on mobile devices, but I did not spell that out. I had left too much work for the audience. First, they had to interpret my chart. Then, they had to relate it to the business problem that they were familiar with.

A better approach is to first tell the audience exactly what the insight is, and then explain how the chart illustrates the insight.

Use the heading of your narrative text to describe exactly what the insight is. Use the text to explain what the chart shows. A good heading will reference a known problem, and grab the audience’s attention. Once engaged, they will be more likely to spend a few seconds to understand the insight presented by your data.

The Narrated Chart

Applying these principles to the Video Play Percentage chart, I redesigned it as shown below. The chart gets across the same information that I was trying to convey previously, but the change in narration makes it much easier for the audience to engage with.

The major changes are:

• Heading that relates the business issue (Video Plays are Low) to the key information conveyed in the chart (Mobile Users Don’t Click “Play”).
• Replacing segments that are indirectly related to the main point (operating systems) with segments that are more direct (Desktop, Mobile).
• Narration that describes the Video Play Percentage metric, describes why it is important, and relates it to the business issue.