By Rama Ramakrishnan

Last week, I had the pleasure of attending two industry events that generated some interesting discussions around big data in retail marketing.

First, I jetted to Chicago to participate on the “How to Effectively Identify, Profile and Track Customers’ Shopping Behavior, to Build A Better, More Customized Shopping Experience” panel at the 2nd Annual Big Data Retail Forum. We had a great group of panelists and an audience full of retailers eager to learn ways of obtaining insight into shoppers’ emotional triggers. One of my fellow panelists, Michael Wilhite of Dunnhumby, discussed how his company is increasingly incorporating survey data to do this at an aggregate level. I then described how CQuotient uses text in product descriptions and customer reviews to infer the emotions shoppers associate with products and product categories. Doing this connects emotional tags directly to individual  customers, making it very actionable.

I was then asked, “Isn’t getting insight from data a lot of work?”

My answer: Mostly yes, but there are interesting opportunities where you can get more benefit with less work!

For instance, when you shift from broadcast emails to segmented emails (even if just a few segments), you experience more benefit but it requires much more work. However, when you go from segmented emails to individualized emails – or “a segment of one” – it can be much more beneficial but much less work, if you use a platform like CQuotient!

There was also significant interest in gathering and using in-store browsing data. Retailers want to understand the new brick-and-mortar sales funnel in order to improve the shopper experience. Unfortunately, the insights from analyzing this data so far (at least the ones that were reported) don’t seem to pass the “so what?” test. For instance, if there is a 40 percent correlation between the amount of time someone spends in the store and the amount of money they spend, do you force people to spend more time in the store? If the back wall of the store gets less traffic than the right or left wall, should you convert the back wall to a front wall? What is the appropriate next step if you determine that certain areas of the store send more traffic to the fitting room? In other words, the key to using big data for creating a better shopping experience is having actionable data.

Later in the week I returned to Boston to participate in the marketing panel at Analytics Week, where we discussed bottlenecks, threats and opportunities for big data within the marketing analytics arena. There were a number of interesting questions posed to the panel, such as how to hire good data scientists and which fields provide the best training for data scientists (my pick: experimental particle physicists! Hunting down the Higgs boson isn’t that different from figuring out if customer will buy a Prada handbag at full price. I swear!).

One of the first questions was: What was the fundamental analytics problem CQuotient had to solve?

My answer: Personalization is easy. Good personalization is really hard.

For retailers outside the grocery and drugstore verticals, two things make personalization very difficult. First, the average number of transactions per customer per year is between one and two. Second, the merchandise may change several times in a year. If you combine these two challenges, all the traditional approaches – such as “people who bought this also bought that” – breaks down completely. CQuotient cracked this problem by bringing in and mining data that nobody else uses for personalization: product descriptions and customer reviews.

We were also asked to name analytics initiatives we tried that did not work out. One of my co-panelists, Bill Simmons of DataXu, talked about building numerous reporting capabilities, including reports and dashboards, into their product and then discovering that hardly anyone really uses them. This is an interesting observation in that it underscores the importance of actionable information.

More broadly, there are two gaps: the data-to-decision gap and the decision-to-action gap. I have always believed that we best serve the needs of retail decision-makers if we think hard about how to bridge the data-to-decision gap and generate decision recommendations from the data (i.e. what should we do next?), rather than simply dump the data into dashboards and further aggravate information overload.

At CQuotient, we take it even further, and execute the actions themselves in real-time, such as when customers open an email, come to the site or ask a store associate for help. In other words, we bridge both the gaps in one shot and connect data to action directly.  Only by connecting data to action directly, will the promise of big data be fully realized.

All in all, I was amazed at the extent of interest in personalization and excited to share my perspective along with other industry experts. These discussions reinforce the need for retailers to provide consumers with tailored experiences based on their individual likes and preferences…and the demand for solutions like ours to help them tackle the big data challenges that will help them deliver what their customers want.