How often should you revisit your KPIs?

I have been thinking lately about the right intervals for revisiting and changing your site and business KPIs. I won’t bore you with all the back and forth, but merely share a few thoughts.

The stability of your business plays a role. If you are an established business, with established goals of your site, revisiting your KPIs too often suggests to me that you didn’t have the right ones in the first place. If, however, you are a newer, (somewhat) flying-by-the-seat-of-your-pants business, perhaps perfecting your approach, I can understand a much more fluid approach to what constitutes “success” and a need to more continuously evolve your KPIs.

My overall thoughts are that KPIs can’t and shouldn’t change every month, even for the latter business example. (Would it be unprofessional of me to say, “duh”?) You should be consistently measuring against the same yardstick.  However, I do think it’s good practice to take a look every three to six months and make sure your KPIs are 1) useful and 2) complete. Do you actually need all of them? Or, on the flip side, is there something new that should be included? Perhaps there are new capabilities you have developed that would allow a new KPI to be measured? After all, a year can allow for a lot of development in the analytics industry. Take advantage of new measurement options.

Recently, I have been involved in the re-evaluation of our KPIs. At the beginning of this effort, the website product team and the analytics team were involved in brainstorming new ways to evaluate the success of the site. Once we decided 1) what we should measure and 2) what we could measure, figured out the overlap of the two and selected from those, analytics began publishing the information. Now, a few months down the track, we’re at a point where our product managers are somewhat comfortable with the information, and the time has come to revisit. (After all, they can’t give feedback on something they don’t even understand or use yet. You have to give the information some time to allow for informed feedback.) Are they using the information? What’s missing? What’s overkill? I expect to do this every 3-12 months from here on out. Just like our site, I expect our measurement of it to be iteratively developed over time.

Parting hint: If you’re not sure if something is helpful or not, try removing it for a month. If no one complains, you have your answer. (But I never do this. Never. No, Really.)

The (most?) valuable trait of analysts (that you can’t teach)

I have been thinking a lot about the type of analyst I enjoy working with, and what I think the critical elements of being a good web analyst are. In the course of doing so, I had an interesting realisation, that I look forward to putting into practice next time I’m searching for an analyst.

We’ve all read a thousand job descriptions, and we know the drill. Attention to detail, analytic skills (of course), able to synthesize large amounts of data to extract meaningful insights, deliver concise message to stakeholders. Etc. Etc. Etc.

But the trait I’ve not seen (often) on job descriptions (or heard in people’s conversations about what they’re looking for) is curiosity.

I want to both be and work with analysts who are curious. Who are forever asking “Why? Why? Why?” Who look at the site redesign of their favourite site, and think, “Oh man I wish I could get my hands on their data, I wonder what they’re seeing …” (And perhaps tries to hack at it via Compete.com or another competitive intelligence source.) I want the analyst who takes the initiative, and may even get a little side-tracked every now and then, because their curiosity takes them down an investigation path that no stakeholder or boss has asked them to go, or even thought to go. (The gems that can come of this …!)

If you’re lucky, you’ve worked with these kinds of analysts. If you’re very lucky, you are one yourself. (FYI, the fact that you’re reading a blog about web analytics pretty much suggests you are that curious person interested in the field. The 9-5 analysts don’t do this …) But me? I want to search for this, to hire and retain for it, and not just as a “nice to have”. This is top of my list. I can teach you how to use Omniture. I can’t teach you how to be interested in what we do, or to be curious to learn and grow.