Are You Boiling the Ocean?

You have just been assigned a new project at work, and you think to yourself, "I am going to be more data driven on this project."  The first thing you do is start daydreaming about all the data you want.  Before you know it, you have a list of reports you want that is longer than War and Peace.  Heck, you don't what you are going to do with all the data, but you want it!  You are now officially boiling the ocean. Your analyst will get all the reports done about the same time as you boil the ocean.

A far more time efficient process is to be hypothesis driven.  You have a hunch about what is going on.  Why not leverage that intuition to reduce your decision making time dramatically? By creating a hypothesis about what the problem might be, you do two things.  First, you limit the data you need to collect.  Second, you pull data that you can take actions on.  Both make you go faster.

Let's say that you have been assigned to improve your account creation funnel.  You have a four step funnel and you have a hunch that step #3 is where you lose everyone.  You can postulate a hypothesis that says that you think that you are losing more people from step #3 than any other step in your funnel.  If the hypothesis is true, the implication is that you should focus on improving step #3 before you work on other steps (all other things being equal).  You now have a fairly straightforward analysis that your analyst should be able to bang out in Google Analytics or a similar tool.

Let's say that your analysis comes back the way you expect.  You now can formulate a hypothesis about what is wrong on step #3.  For example, I think that customers are getting confused about what to input into field XYZ.  You can now use a tool like Mouse Stats to see if customers are pausing longer on field XYZ or exiting on that field.  If they are, you know where to focus your actions.

At the end of the day, making rapid improvements to your site or app is the name of the game.  Boiling the ocean is your worst enemy when trying to make rapid improvements.  Let hypotheses guide your actions and forever be more efficient!