A heartfelt thank you

On Tuesday, 15 March 2011 the WAA held the inaugural Awards Gala. (For the record, it’s pronounced gah-la. Not gay-la.) It was a wonderful event – a chance to spend time with the lovely people in this industry, and make new friends. The Gala was, in my opinion, a hugely successful experience – I can’t wait to attend the next one.

As a part of the Gala, the WAA handed out the very first Awards of Excellence. The categories were:

Client/Practitioner of the Year
Most Influential Agency/Vendor
Most Influential Industry Contributor
Web Analytics Rising Star
Innovator/Technology of the Year

I myself was so incredibly humbled just to be nominated for the award of Web Analytics Rising Star, let alone become a finalist. Those two alone seemed to good to be true.

Imagine my shock to win …

So, from the bottom of my heart,  thank you.

Thank you first and foremost to the kind soul who even thought of nominating me. Thank you for the WAA members who kindly voted for me to be a finalist. And thank you to the Awards judges, who made such a difficult choice from an amazing list of finalists, who are all so deserving.

I have been working in web analytics for a few years, but really only got involved with the community last May. I have loved every minute I’ve spent getting involved with the WAA, the Analysis Exchange and #measure. I learned more in six months than I had in years prior.

So all I can say is thank you. I really, truly love this community of amazingly smart people. Thank you for welcoming me so generously into it. Thank you for letting me learn with you and from you. Thank you for the time we’ve spent discussing, debating and encouraging each other.

Congratulations to the award nominees, the award finalists and the award recipients. You are what makes this community such an amazing thing to be a part of. I am humbled and grateful for the award, but more thankful still to be a part of such a wonderful community.

WAA Award

Statisticians + Web Analysts = Awesomeness

One thing I have found can work very successfully is a hybrid team of web analysts and statisticians. When you combine the business and website knowledge that the analyst has with the “mad stats skills” that the statistician brings, you can create some truly powerful work.

There are a lot of different things that a web analytics team can leverage a statistician’s help for. This is by no means an exhaustive list, merely a place to get started.

1. Significance Testing

So you’ve run an A/B or Multivariate test. While your testing tool will likely also advise of you the statistical significance of your results, a statistician can dive deeper into this, and help you to measure significance outside of your tool. Perhaps you noticed shifts in site areas that weren’t one of your test success measures – a statistician can help you decide if these are merely interesting, or statistically significant.

Or perhaps you’ve tested in more of a time-series fashion. A statistician can try to tease out whether the change had an impact, or whether changes are due to seasonality. (This relates closely to the idea of an Impact Analysis.)

2. Impact Analysis

You make a site change, and you notice an increase in visits to a site area, or some key metric. You’re tempted to attribute this entire shift to the site change. (“Woo hoo! We’re up 5%!”) However, what about changes in marketing spend? Seasonality of your site traffic? Social initiatives? Are you taking those into account before reaching your conclusion?

A statistician’s analysis can attempt to tease out those additional variables to estimate the impact of the actual site change, vs. these confounding variables.

This same approach can be used to measure the impact of industry events or company changes (outside of the website) – anything, really. The benefit here is a better understand of the actual impact of events or initiatives, but a nice perk should be presenting your findings to the business and not having to freeze like a deer in headlights if someone says, “Yes but we spent another million dollars in paid search last week – did you factor that in?”

3. Standard reporting automation

Statisticians can use tools such as SAS to fetch data from FTP, combine and compute it, and deliver outputs to your system of choice (for example, Excel, if that’s somewhere you’re comfortable working.) This can allow you to schedule FTP delivery of SiteCatalyst reports, Discover reports, ad server reports (etc) – basically data from multiple sources – have SAS do the work of fetching multiple data sets, combine them and output to Excel.

That, however, doesn’t mean you need to deliver a huge scary data sheet to the business. On top of the data, you can build  a more user-friendly view (preferably formula-driven, so that you’re not manually updating!) in Excel to present the data.

This allows you to take a lot of the manual part (copy-paste, copy-paste) of standard reporting out the equation, and focus your time on explaining the shifts you might be seeing in the report. e.g. Perhaps traffic to a specific content area is down – start digging in. What traffic sources are driving it? Are there particular pages experiencing a more dramatic shift?

In addition, once the business sees the value of this work (the time it frees up for analysts to actually analyse!) it may actually help  argue for further automation and investment in further tools. So make sure you provide those insights, and use this work to prove why you shouldn’t spend your time copy-pasting.

4. Forecasting

Statisticians can build forecasting models to predict your site traffic, sales, ad impression volume – pretty much anything. You can go short-range, or long-range. Perhaps a simple “forecast through end of month” will suffice to start, or maybe you want to start forecasting three or six or twelve months in advance.

So why would you do this? Well, good analysts know that data needs context. That’s why we have KPIs, or compare month over month, year over year – to understand whether “2.6%” is “good”. Comparing to a forecast can be another way to get context for your data. If you’re diverging from your forecast, you can start digging in to see why. This divergence might be good – perhaps you saw a better than expected responses to your marketing initiatives. But on the flipside, you might also need to frantically search for why you’re suddenly down -10% compared to forecast …

Even a through-end-of-month forecast can be helpful here. An EOM forecast will tell you where you’ll likely end the month, based on current performance – even though you’re only on day 9. This will allow you to course correct throughout the month, rather than waiting till end of month to realise you didn’t match your forecast.

If your business sets site goals, forecasts can be the first step. First, forecast where your business will be for the next twelve months without any major initiatives. Simply assume the status quo. Then, look at the initiatives you want to add on top of that, and assess how much of an impact they may have. Forecast + specific initiatives = your goal. A statistician can also help you look back over time at previous initiatives and analyse their impact, to make sure that you’re not overstating how big an impact something new may have. (How many times have you heard “This is a game changer!” and found it barely moved the needle?)

There are still things you need to keep in mind when forecasting, but even starting small can bring value to your business.

Group Hug!

Still, analysts and statisticians may sometimes face some hurdles. Analysts need to learn the language of statisticians, and statisticians need to either learn the business, or be guided  by the analysts. A statistician exploring data with no understanding of the business, the website, or what any of it means normally doesn’t reveal great insights. On the flip side, the analyst really needs to start learning and at least dabbling in the world of statistics, and be able to translate complex concepts for the business users you support.

However, a cohesive team that learns to work together and leverage each other’s strengths can do amazing things.

Don’t have access to a statistician? Students often need real-life data for school projects. Consider seeking one out! (Who knows – you might find yourself a great future employee.)

Omniture Summit 2011 on Twitter (Day 1)

So, because I’m a huge nerd (and I assumed others might be too) I thought folks might enjoy some information on #omtrsummit (aka the Adobe Omniture Summit 2011) on Twitter.

Half way through the opening session today (I’d say around 9.30AM Utah time) I started a hashtag archive using Twapper Keeper.

Some completely fun but not very actionable findings:

Approximately 17% of Summit Attendees tweeted: 441 unique usernames tweeted at least once, compared to 2600 attendees. (Note: I’m sort of assuming that if you didn’t tweet in the first day, you’re not likely to throughout the rest of Summit, but I’ll gladly check those findings on Friday!)

Top 10 Tweeters, in order of volume of total tweets:


Total 10 Tweeters, excluding retweets/via:


Oh yeah – and 1.4% of tweets on Day 1 included a reference to Charlie Sheen.


Social Media Analytics: Moving From Engagement to Measurement

[Originally published at The Review]

It’s no mystery that social media has been the new buzzword of the past few years. However, companies are quickly moving from “Gee, we really should be doing social” to “Now, how do we measure it?” There are a number of ways a company can begin measuring their social media efforts and those include:

1. Measuring the effect of social media efforts within the network itself

2. Tracking social media links back to your website

3. Understanding social media in the context of other initiatives

Measuring within the network itself

Analysis of Impact and Engagement

Step one of measuring your social media initiatives is to measure success within the network. While there are a wide variety of social networks, we’ll focus on Facebook and Twitter as the primary two.

Measurement of Facebook might include monitoring number of fans, fan demographics, fan interaction with posted content (comments, likes), organic fan posts and traffic to Facebook page, or use of a Facebook app. Facebook analytics can come from Facebook Insights, but there are also options to add the code from your web analytics solution to your Facebook pages.

Twitter has its own set of tools. Two popular ones are Klout and Twitalyzer. Klout combines thirty-five different variables into one “Klout” score: a measure of social influence. While the variables behind Klout score are intentionally hidden to avoid “gaming” the system, one downside is that the lack of visibility makes it hard to understand what’s driving your Klout score – or how to increase it.

Twitalyzer, on the other hand, provides transparency into all their calculated metrics. For any compound metric, a user knows exactly what is going into this score. For example, “Impact” is based on number of followers, mentions, retweets and post frequency. What’s more, Twitalyzer provides users with data-driven recommendations for how to increase their scores.

Other Twitalyzer measures include number of followers, number of lists you are on, number of mentions or retweets, plus calculations of both potential and effective reach: how far your tweet may reach within the network. Twitalyzer also offers users the option to tailor their report to see only metrics of interest to them, as well the ability to set goals. Other benefits include a visual network map to explore your connections, a comparison tool to compare your scores to other Twitter users and customizable sentiment analysis.

So why would you measure your impact within Twitter or Facebook? Social media is more than just a broadcast network – engagement matters. By measuring more than just fans or followers, you can begin measuring your success in engaging with consumers.

Search or Hashtag Analysis

Tools like The Archivist and Twapper Keeper allow you to build an archive of a particular search or hashtag. The Archivist even provides you a dashboard view of top contributors to a hashtag, tweet volume by day and top words used. However some tools (Twapper Keeper and Tweetake) will actually allow you to export full Twitter content, for offline analysis in Excel, SPSS, SAS or any other data analysis or exploration solution. This offline analysis allows for rich time/date and textual analysis of Twitter conversions.

These types of analyses can tell you what time of day a community tends to tweet, and allow off-line, more robust sentiment analysis. These insights allow for tailored posting schedule and contents, to best suit the audience.

Measuring social media back to your website

Measuring your social engagement within the network is a great start. But if your social efforts don’t result in traffic, sales or leads, it’s hardly a justifiable effort.

The easiest way is to leverage the analytics that your URL shortener provides. When social media links are posted, they are typically shortened (to save characters) through services such as bit.ly. These services provide data about how many clicks you received to each link. However, that’s where it ends. A click tells you only that: that the visitor clicked the link. It doesn’t tell you what they did after that. Did they close the browser window before it even loaded your site? Did they see one page of your site then leave? Or did they actually engage with your content, and perhaps funnel through into an online sale?

That’s where campaign tracking comes in. Using the same methods of campaign tracking used for other online initiatives, you can track your social media behavior back to your website.

Each web analytics tool does campaign tracking a little differently, so it’s worth touching base with your marketing or web analytics team to see how to set this up for your solution. For Google Analytics, campaign tracking involves appending campaign variables, in a specific format, at the end of the URL for Google Analytics to read. (Note of course that this is only an option when your social post contains a link.)

Without campaign tracking, you might post a Tweet and link back to:


Campaign tracking would involve adding variables at the end of the URL:


source=twitter  tells you the source of the traffic. In this case, this link was posted to Twitter.

medium=social  tells you that this was a social media post (vs perhaps an online media or PPC link.) Think of this as representing the “channel”.

content=nutrition  tells us that we’ve categorized this post as a “nutrition” related. Content allows you to group “types” of posts. (For example, quizzes vs nutrition vs recipes.)

campaign=freedietbook gives you a short description of what the post was. It should be short, but enough for you to recall the post.

To create these campaign codes, you simply use Google’s campaign tracking code generator. Based on your inputs, it will auto-generate the URL with campaign tracking. This new URL is then fed into your bit.ly or other URL shortener, and carries through campaign information into your web analytics solution.

So why do you need this?

This campaign tracking will allow you to compare different mediums (for example, Twitter vs Facebook) and the quality of traffic they drive. By categorizing posts into different “content” groups, you can analyze how different types of posts drive traffic and behavior, and even look at how visitors driven from one particular post behave. This includes looking at whether these visitors leave your site, stay and engage, how many page views they see, how much time they spend, and whether they convert into a lead or a sale.

This information can help to optimize future posts. For example, if recipe posts convert well into sales of a book, a business may focus on more of these posts. You can even compare multiple of elements: for example, do recipe posts perform better on Twitter or Facebook? Add in day of the week and time analysis, and you have a rich analytics opportunity to provide insights for future posts.

Campaign tracking even allows for calculation and optimization of ROI. If you know the time that is spent on social media efforts, and the actual sales driven by traffic through those posts, you can calculate that return on investment.

Understanding social media in the context of other initiatives

So by now you’re measuring your impact and engagement within the Facebook or Twitter community, as well as the behavior of social media traffic back to your website. However, couldn’t social media have an impact even without someone engaging with your brand or visiting your website?

Coupons are a clear way of tracking a social media promotion back to sales, as a specific coupon code can be used to distinguish between different channels.

Loyalty cards may be leveraged, if you can entice consumers to couple their loyalty card with their social media identity.

Another approach is to simply ask. Customer surveys can help here. The “Where did you hear about us” may help tie at least a portion of your offline sales to social media, though there will likely remain a segments of customers who choose not to answer, or who don’t necessarily recall when or why they first decided to purchase your product.

You may also analyze the correlation between social media efforts and sales to tell you directionally whether your efforts are working.

Finally, don’t forget that it can often be a culmination of different initiatives that result in a visit to your site, or a customer walking into your store. A customer may engage with your brand through social media, then later visit your site through a paid search link to research a product, and ultimately purchase offline. Television, radio, print or banner ads may be further mixed into this, requiring some pretty serious multi-channel analytics efforts.

Get Started

The next step in social media is to move from simply getting involved to measurement. While social is a new channel, it is not so uniquely different that we can’t leverage learnings and best practices from other channels to help us understand its impact. Leveraging social media analytics tools, campaign tracking, and multi-channel efforts can help you understand the impact of your social media efforts.