Want to Improve Your Paid Ads? Get out of Your Ivory Castle!

Last blog post I argued that it was pretty hard to become a slave to data.  While I strongly believe that, I also believe that sometimes you have to put your analytical tools aside and get out into the real world.  Leave your ivory castle!

Paid advertising teams often are very analytical folks.  They speak in the language of CTRs (click through rates), CPAs (cost per acquisitions), view through conversions (conversions that came from folks who saw ads but didn't click on them), etc.  Many spend their days analyzing numbers, moving money across keywords, ads, campaigns, etc.  I too spend a tremendous amount of time looking at these metrics and often say that my job is akin to being a portfolio manager.  I look at numbers much of the day and move money around between projects to optimize our spend.

However, I feel strongly that marketing folks, and in particular paid advertising types, need to go out into the real world and mix it up with customers.  One of my favorite ways to do that is to go to tradeshows.  I try to go to roughly only show a month.  People (especially my wife) often ask me, "Why are you going to so many shows?  Don't you have people who can run the shows for you?"  Of course I have very competent team members who can run the shows.  However, I go to listen to our customers.

I go to the booth and ask questions.  I often don't do a lot of talking myself.  Here are some of the questions I ask:

To Existing Customers:

• What do you like best about the product?
• How did you find out about us?
• What other solutions did you consider?

To Potential Customers:

• What are the biggest challenges you face today?
• What options are you considering to address those challenges?
• How are you researching those options?

My favorite scenario is to have one or more existing customers in the booth try to sell some potential customers.  This happens a lot with TeamSnap because customers generally love the product and are pretty vocal about singing our praises.  I just sit back and listen to how the existing customers sell the potential customers.  They often can really crisply describe the benefits of the product.  Best of all, they use language that potential customers understand. They don't have the problem of being too close to the product.

During these types of interactions, I just sit there and take notes.  After the show I huddle with the folks working with me on our paid ad programs.  We almost always come up with new campaigns for AdWords, Facebook, etc.  As well, we typically come up with alternate versions of existing ads that more succinctly describe the product benefits.  These new campaigns and creatives generate lots of new analyses and learnings.

The more varied your customer base, the more you need to get out and meet folks.  TeamSnap is used in over 100 different sports.  As well, we have customers in 196 countries.  As such, there are a ton of ways customers use the product.  An adult hockey team in Vancouver has very different needs than say a youth travel soccer team in L.A. or a high school rugby club in London.   You cannot pick up those sort of differences from behind a desk - sometimes there is just no substitute for leaving the ivory tower.

Can You Become a Slave to Data?

In a recent discussion with my colleagues about data-driven decision making, one of them asked, "Can you become a slave to data?  In other words, can you become so focused on the numbers that you ignore common sense and make bad decisions?"  It was a great question.  After a little reflection, I responded that yes in theory you can be a slave to data.  However, in the real world I struggle to think of companies that truly have become slaves to data.

On a scale of 1-10 (where 1 means totally winging it and 10 means being completely data driven), most companies are much closer to 1 than 10.  When other Internet professionals ask for my advice on issues they are facing, I am often shocked by how little analysis they have done.  I often will say something like, "That is a good issue.  I bet looking at metrics X, Y, and Z will tell you your answer.  I am sure you have looked at X, Y, and Z, right?"  Usually I am greeted by a long pause and a lot of foot shuffling.  If an average company is a 2 or 3 on the data driven scale, they can easily move up a number or two (or more) and not become slaves to data.

Secondly, if you are using a hypothesis driven approach as I talked about in my last blog post, you already are bringing intuition and common sense into the equation.  You are only running analyses on things that you have already thought about.

Lastly, if you really understand the limitations of the common analytical tools, you know that there is a pretty large margin of error built in.  Let's take the free version of Google Analytics (GA).  Try running a report one way and then run the same report a different way.  The results are often a little different, even when GA doesn't say it is sampling the data.  When GA says it is sampling the data, the variance between reports can be pretty large.  Alternately, hook up the GA e-commerce tracking to your website and compare how many transactions GA gets to how many your e-comm platform says you actually received.  You will often see that GA misses about 15% of the transactions.  Drill down on some of the transactions in GA and you will find garbage transactions.

I picked on Google Analytics, but you can find similar limitations in most of the tracking tools.  Let's not even get into attribution modeling.  I can barely explain to you what made me buy product A over product B.  How is a relatively simple attribution algorithm able to describe what drove my buying behavior?  It can't.

If you understand these limitations in today's tools, you realize that our tools are best used as directional indicators.  They cannot give you exact answers, but they sure can point you in the right direction.  Your brain must be fully engaged at all times - you cannot blindly trust the numbers that are spit out.  I often like to triangulate my results using a couple different methods or tools to increase my confidence in my measurements.

The bottom line is that if you have a solid analytics team, the chances are extremely slim that you have to worry about becoming a slave to data.  Whew, you can cross that off your list of things to worry about!

Are You Boiling the Ocean?

You have just been assigned a new project at work, and you think to yourself, "I am going to be more data driven on this project."  The first thing you do is start daydreaming about all the data you want.  Before you know it, you have a list of reports you want that is longer than War and Peace.  Heck, you don't what you are going to do with all the data, but you want it!  You are now officially boiling the ocean. Your analyst will get all the reports done about the same time as you boil the ocean.

A far more time efficient process is to be hypothesis driven.  You have a hunch about what is going on.  Why not leverage that intuition to reduce your decision making time dramatically? By creating a hypothesis about what the problem might be, you do two things.  First, you limit the data you need to collect.  Second, you pull data that you can take actions on.  Both make you go faster.

Let's say that you have been assigned to improve your account creation funnel.  You have a four step funnel and you have a hunch that step #3 is where you lose everyone.  You can postulate a hypothesis that says that you think that you are losing more people from step #3 than any other step in your funnel.  If the hypothesis is true, the implication is that you should focus on improving step #3 before you work on other steps (all other things being equal).  You now have a fairly straightforward analysis that your analyst should be able to bang out in Google Analytics or a similar tool.

Let's say that your analysis comes back the way you expect.  You now can formulate a hypothesis about what is wrong on step #3.  For example, I think that customers are getting confused about what to input into field XYZ.  You can now use a tool like Mouse Stats to see if customers are pausing longer on field XYZ or exiting on that field.  If they are, you know where to focus your actions.

At the end of the day, making rapid improvements to your site or app is the name of the game.  Boiling the ocean is your worst enemy when trying to make rapid improvements.  Let hypotheses guide your actions and forever be more efficient!