Digital Experiments and Useful Analytics Must Go Hand-in-Hand

Mac Slocum
May 20, 2008 | Permalink | Comments (0) | Listen

Experimentation without analysis isn't worth much.

It's a succinct and obvious point, but this one phrase was my biggest takeaway from the recent IDPF Digital Book event. Leslie Hulse, vice president of digital business development at HarperCollins, drove home the experimentation-analysis relationship while discussing one of HarperCollins' free audio download experiments. Hulse concluded:

The marketing people would say this was very successful because it got all kinds of attention, but we really didn't see an impact on sales. It wasn't linked to registration, so we didn't feel we got much out of this in terms of something we could use down the road. What we learned from this experiment was: when it's free audio with no DRM and no registration, it's too easy to take it and run. So we need to tie it to registration or use DRM or use a watermark so that we can contact these people in the future or know more about how they're using it.

The merits of DRM are debatable, but Hulse's broader point is important to consider: An experiment launched without a measurement device isn't really an experiment; it's a blind giveaway.

As Hulse noted, a simple registration form can provide baseline metrics and contacts for future products. For more in-depth information, advanced analytics tools (even free ones) can be integrated into experiments to track downloads, page views, unique visitors, user-session times, geographic targeting, forwards/emails and other social components. Privacy needs to be considered in any tracking effort, but a little common sense and planning can easily find the sweet spot between consumer comfort and detailed data points.

"The key thing for us is that all these experiments must be measurable," Hulse said. "We're trying to do things where we can measure the results and move from there."

Related Stories:

• BISG Study: Publishing Experimentation Catching On
• It's Time to Accept an Ambiguous Digital Fate
• A Comeback for DRM?