10 Problems with Foursquare Big Data Prediction of iPhone 6s Sales

crystal_ballFoursquare, an App that tracks your movements and checkins, made a prediction on launch weekend iPhone 6s sales. It is a good PR to latch on to a major event to showcase their new entry into data analysis. This is also going to be used as an example of Big Data analysis. Since Foursquare has access to time series data on locations of its users, it is argued that they can put that data to work to “unlock value from it”.  Combine this movement with location of stores, you have foot traffic information on stores which they believe is a powerful dataset.

Here is how Foursquare made the prediction,

So our Place Insights team looked into foot traffic at Apple stores leading up to the launch of the iPhone 5, 5S, and 6 and analyzed it alongside Apple’s public sales data.

Combining Foursquare’s foot traffic data with Apple’s sales data on a graph shows how closely the two are linked. Visit growth is clearly a strong general advance indicator for sales performance of the launch weekend.

Foursquare predicts that launch day foot traffic will be about 360% of a typical Friday. This likely means that Apple will sell 13–15 million iPhones this weekend,

1*tMvRSHrP5FIUX10tHC77bQBut is the method valid? Is the prediction trustable? The answer is NO to both questions. This is yet another case of chasing Big Data and letting spurious correlations make seemingly correct predictions. Here are the problems with Foursquare’s method and prediction:

  1. Does foot traffic mean sales? Isn’t it possible more simply visit the store during launch weekend to look at the new phones?
  2. What percentage of people who walk into Apple stores have Foursquare app? Is that a representative sample.
  3. What other factors will drive foot traffic? This time there is iPad Pro which is going to bring in more people who may want to see it in action.
  4. Problem with iPhone 5s data used by Foursquare. iPhone 5s was not offered for preorder and did not put up big numbers in its initial quarter.
  5. What percentage of iPhone sales happen in Apple stores? Even if we accept point 1 above we should scope the effect of Apple stores on total sales. According to a 2012 survey,  Apple Stores accounted for just 21% of the iPhone Sales. Even if this is different for opening weekend that explains only portion of sales.
  6. More sales are shifting online. Apple said the online demand for iPhone 6s Plus has been exceptionally strong and exceeded their own forecasts for the preorder period.
  7. More sales happening though other channels. T-Mobile CEO stated iPhone 6s pre-orders on T-Mobile are up an impressive 30% compared to iPhone 6 pre-orders last year.
  8. Changes in pricing model affect sales. This year subscription pricing for iPhone 6s is available from all carriers and even from Apple. There is a price war on monthly subscription fee between T-Mobile and Sprint. These factors drive new customers that is going to increase launch weekend volume.
  9. China. China China. Last two times Apple did not launch in China at the same time. iPhone 6s was available in China at the same time as US.
  10. Finally there is no information on confidence level or likelihood information in the prediction. I give them credit for predicting a range vs. one absolute number. But they do not state at what confidence level. That is the difference between big data analysis vs. statistical analysis.

What we have here is a faulty method that falls apart despite Foursquare’s access to largest database of information on the foot traffic of people around the globe. All that big data does not help them see Madagascar from San Diego.

Could the rest of us make better predictions without the Big Data? Yes we can if we do scenario analysis with the new factors that affect iPhone 6s sales. And we can state not only a range but also at what confidence level.

Big Data has applications but this is not it. Foursquare has bigger ambitions to monetize all their big data by making similar predictions for Marketing, Real Estate, Finance and Credit Scoring. All these predictions will suffer from the same challenges unless you build a more comprehensive model with foot traffic data as just one part of prediction. Otherwise we will be looking for Shamu in Madagascar.